-- Logs begin at Sun 2022-10-30 00:05:02 UTC, end at Sun 2022-10-30 01:19:10 UTC. -- Oct 30 00:34:06.351814 dv-cinder-861491-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 30 00:34:06.363944 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 30 00:34:06.366565 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Oct 30 00:34:06 2022] *** Oct 30 00:34:06.366565 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 30 00:34:06.366565 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 30 00:34:06.366565 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: nodename: dv-cinder-861491-6-iscsi Oct 30 00:34:06.366565 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: machine: x86_64 Oct 30 00:34:06.366565 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: clock source: unix Oct 30 00:34:06.366565 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: pcre jit disabled Oct 30 00:34:06.366565 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: detected number of CPU cores: 4 Oct 30 00:34:06.366565 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: current working directory: / Oct 30 00:34:06.366565 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: detected binary path: /usr/bin/uwsgi-core Oct 30 00:34:06.367329 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: your processes number limit is 47680 Oct 30 00:34:06.367329 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: your memory page size is 4096 bytes Oct 30 00:34:06.367329 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: detected max file descriptor number: 2048 Oct 30 00:34:06.367329 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: lock engine: pthread robust mutexes Oct 30 00:34:06.367329 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: thunder lock: enabled Oct 30 00:34:06.367329 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 30 00:34:06.367329 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 30 00:34:06.406294 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: Python main interpreter initialized at 0x55b370f9f9c0 Oct 30 00:34:06.406294 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: python threads support enabled Oct 30 00:34:06.406294 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: your server socket listen backlog is limited to 100 connections Oct 30 00:34:06.406294 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: your mercy for graceful operations on workers is 90 seconds Oct 30 00:34:06.406706 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: mapped 403077 bytes (393 KB) for 2 cores Oct 30 00:34:06.406782 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: *** Operational MODE: preforking *** Oct 30 00:34:06.407008 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: *** uWSGI is running in multiple interpreter mode *** Oct 30 00:34:06.407008 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: spawned uWSGI master process (pid: 88190) Oct 30 00:34:06.407223 dv-cinder-861491-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 30 00:34:06.408837 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: spawned uWSGI worker 1 (pid: 88200, cores: 1) Oct 30 00:34:06.408837 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: spawned uWSGI worker 2 (pid: 88201, cores: 1) Oct 30 00:34:06.408837 dv-cinder-861491-6-iscsi devstack@keystone.service[88190]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 30 00:34:10.794149 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 30 00:34:10.794149 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: from cryptography.utils import int_from_bytes Oct 30 00:34:10.794963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 30 00:34:10.794963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: from cryptography.utils import int_from_bytes Oct 30 00:34:10.797050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 30 00:34:10.797050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: from cryptography.utils import int_from_bytes Oct 30 00:34:10.797509 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 30 00:34:10.797509 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: from cryptography.utils import int_from_bytes Oct 30 00:34:10.845829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 30 00:34:10.846042 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 30 00:34:10.846192 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] command line args: [] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 30 00:34:10.846466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 30 00:34:10.846647 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ================================================================================ {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 30 00:34:10.846898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] admin_token = **** {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.847145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] config_dir = [] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.847332 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.847513 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] config_source = [] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.847692 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] debug = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.847892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.849348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.849348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.849348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.849348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.849656 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 30 00:34:10.849737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] list_limit = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.849818 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 30 00:34:10.849944 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] log_config_append = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.850066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] command line args: [] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 30 00:34:10.850189 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.850189 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] log_dir = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.850189 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] log_file = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.850538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 30 00:34:10.850659 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.850659 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.850899 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ================================================================================ {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 30 00:34:10.851034 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.851178 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] admin_token = **** {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.851316 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.851316 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.851568 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] config_dir = [] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.851705 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.851821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.851821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] config_source = [] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.852056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.852238 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] debug = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.852344 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.852424 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.852532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.852612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.852688 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.852760 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.852836 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.852914 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.852991 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.853068 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.853145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.853221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] list_limit = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.853297 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.853378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] log_config_append = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.853378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.853620 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.853719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] log_dir = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.853719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] log_file = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.853719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.854013 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.854013 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] publish_errors = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.854206 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.854280 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.854395 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.854480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.854553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.854671 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.854772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.854772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.854973 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.854973 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.855176 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.855257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] standard_threads = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.855334 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.855406 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.855532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.855613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.855684 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.855804 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.855884 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.855961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] use_journal = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.856043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.856132 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] use_json = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.856236 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] use_stderr = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.856381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.856466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] use_syslog = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.856579 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.856653 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.856752 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.856867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.857098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.857242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.857361 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.857473 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] publish_errors = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.857587 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.857809 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.857936 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.858050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.858050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.858263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.858378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.858499 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.858613 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.858732 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.858861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] standard_threads = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.859058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.859215 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.859345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.859469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.859585 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.859728 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.859832 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.859930 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] use_journal = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.859930 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] use_json = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.860159 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.860275 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] use_stderr = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.860375 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.860471 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] use_syslog = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.860565 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.860679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 30 00:34:10.860783 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.860880 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.860974 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.861068 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.861168 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.861272 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.861272 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.861490 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.861604 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.861717 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.861829 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.861951 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.862078 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.862200 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.862321 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.862435 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.862549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.862672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.862787 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.862900 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.863014 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.863127 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.863240 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.863430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.863546 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.863661 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.863780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.863894 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.864015 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.864234 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.864360 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.864482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.864603 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.864725 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.864895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.865019 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.865135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.865254 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.865368 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.865482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.865482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.865817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.865939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.865939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.865939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.866266 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.866389 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.866504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.866618 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.866732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.866850 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.866850 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.867075 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.867191 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.867310 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.867430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.867545 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.867545 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.867768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.867884 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.867997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.867997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.868223 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.868327 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.868436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.868572 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.868680 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.868777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.868873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.868974 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.869145 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.869245 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.869340 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.869436 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.869531 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.869625 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.869729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.869835 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.869931 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.870025 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.870126 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.870229 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.870352 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.870499 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.870618 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.870618 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.870887 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.870988 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.871092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.871189 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.871294 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.871389 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.871484 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.871591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.871703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.871801 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.871895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.872022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.872224 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.872358 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.872477 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.872595 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.872712 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.872822 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.872936 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.873051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.873169 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.873284 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.873403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.873523 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.873640 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.873761 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.873878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.873993 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.874123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.874276 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.874434 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.874552 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.874668 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.874805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.874917 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.875030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.875145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.875259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.875374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.875499 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.875617 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.875736 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.875861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.875977 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.876105 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.876236 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.876354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.876468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.876582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.876702 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.876818 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.876940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.877095 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.877210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.877325 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.877439 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.877552 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.877676 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.878337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.878458 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.878583 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.878700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.878821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.878937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.879051 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.879165 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.879288 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.879412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.879528 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.879642 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.879756 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.879869 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.879992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.880144 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.880263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.880378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.881113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.881267 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.881531 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.881644 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.881742 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.881839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.881935 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.882032 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.882136 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.882244 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.882380 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.882499 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.882615 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.882722 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.882836 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.882836 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.882836 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.883162 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.883276 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] auth.external = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.883390 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.883504 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.883628 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.883628 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.883851 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.883968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.884101 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.884101 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] auth.password = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.884334 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.884451 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] auth.token = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.884568 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.884688 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.884804 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.884804 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.885027 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.885144 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.885259 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.885407 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.885534 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.885653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.885768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.885892 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.886017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.886134 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.886256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.886378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.886495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] credential.caching = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.886611 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.886736 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.886853 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.886969 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.887076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.887189 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.887304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.887428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.887545 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.887660 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.887776 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] auth.external = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.887891 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.888007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.888575 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.888690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.888791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.888888 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.888985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.889082 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] auth.password = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.889190 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.889290 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] auth.token = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.889388 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.889491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.889598 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.889703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.889811 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.889916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.890014 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.890111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.890207 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.890303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.890411 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] federation.caching = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.890510 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.890607 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.890703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.890800 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.890903 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] credential.caching = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.891010 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.891135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.891251 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.891367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.891483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.891599 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.891723 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.891841 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.891957 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.892072 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.892206 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.892321 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.892448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.892566 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.892682 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.892807 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.892923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.893039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.893162 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] identity.caching = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.893278 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.893393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.893508 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.893623 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.893737 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.893861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.893977 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.894100 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.894217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.894332 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.894445 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.894568 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.894684 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] federation.caching = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.894799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.894914 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.895028 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.895212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.895331 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.895446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.895561 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.895676 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.895792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.895916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.896775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.896905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.897006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.897103 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.897202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.897342 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.897492 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.897595 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.897695 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.897793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.897898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.898011 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.898111 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.898209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] identity.caching = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.898307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.898307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.898307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.898584 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.898684 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.898802 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.898919 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.899036 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.899152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.899278 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.899397 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.899513 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.899628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.899743 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.899858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.899982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.900114 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.900234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.900351 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.900467 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.900583 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.900712 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.900830 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.900946 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.901061 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.901177 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.901298 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.901424 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.901541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.901657 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.901772 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.901888 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.902003 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.902127 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.902246 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.902362 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.902478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.902594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.902709 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.902838 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.902955 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.903071 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.903071 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.903286 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.903408 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.903534 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.903651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.903765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.903880 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.903995 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.904144 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.904276 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.904393 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.904510 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.904625 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.904741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.904856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.904983 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.905100 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.905216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.905331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.905447 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.905563 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.905687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.905812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.905929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.906045 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.906161 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.906284 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.906402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.906524 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.906642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.906757 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.906872 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.906996 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.907113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.907229 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.907344 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.907459 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.907578 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.908020 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.908161 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.908161 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.908382 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.908498 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.908628 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.908810 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.908933 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.909049 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.909165 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.909281 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.909397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.909521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.909639 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.909757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.909873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.909873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.910091 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.910091 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.910318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.910318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.910535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.910651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.910766 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.910891 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.911009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.911009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.911227 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.911342 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.911458 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.911581 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.911788 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.911967 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.912207 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.912332 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.912449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.912449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.912684 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.912801 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.912918 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.913034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.913150 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.913275 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.913394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.913510 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.913626 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.913626 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.913626 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.913626 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.914058 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.914174 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.914289 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.914405 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.914520 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.914645 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] resource.caching = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.914763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.914879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.914996 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.915112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.915227 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.915351 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.915469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.915585 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.915700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.915816 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.915932 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.916057 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.916193 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.916309 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.916436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.916553 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.916668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.916793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.916911 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.917026 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] role.caching = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.917142 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.917257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] role.driver = sql {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.917372 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.917496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.917613 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.917728 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.917842 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.917958 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.918074 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.918199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.918316 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.918432 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.918547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.918663 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.918771 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.918959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.919078 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.919194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.919309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.919424 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.919539 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.919663 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.919780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.919895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.920010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] resource.caching = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.920181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.920313 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.920431 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.920547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.920662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.920777 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.920777 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.921001 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.921119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.921240 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.921420 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.921544 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.921662 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.921787 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.921905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.922022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.922137 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.922253 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.922370 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] role.caching = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.922495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.922614 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] role.driver = sql {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.922729 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.922845 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.922960 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.923075 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.923199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.923316 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.923417 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.923531 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.923647 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.923763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.923887 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.924005 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.924155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.924273 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.924423 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.924542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.924671 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.924789 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.924905 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.925027 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.925144 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.925260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.925384 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.925502 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.925618 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] token.caching = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.925734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.925843 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.925957 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.926081 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.926081 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.926300 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.926415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.926532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.926656 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.926773 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.926889 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.927005 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.927120 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.927235 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.927455 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.927626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.927748 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.927866 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.927982 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.928114 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.928247 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.928375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.928493 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.928611 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.928728 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.928844 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.928844 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.929073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.929189 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.929306 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.929306 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.929306 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.929306 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.connection = **** {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.929748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.929748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.929977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.930093 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.930209 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.930325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.930440 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.930565 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.930683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.930800 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.930916 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.931033 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] token.caching = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.931155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.931287 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.931405 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.931521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.931637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.931755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.931871 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.931995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.932148 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.932267 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.932383 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.932535 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.932684 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.932817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.932943 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.933061 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.933177 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.933293 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.933418 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.933536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.933652 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.933768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.933886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.934002 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.934126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.934126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 30 00:34:10.934346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.934346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.934346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.934346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.934346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.934346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.connection = **** {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.934346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.934346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.935209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.935209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.935209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.935209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.935637 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.935637 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.935855 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.935970 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.936185 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.936422 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.936667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.936900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.937146 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.937380 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.937619 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.937853 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 30 00:34:10.938022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 30 00:34:10.964233 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:10.964469 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:10.964697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:10.968994 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:10.969228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:10.969458 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:10.973287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:10.978488 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.002106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.002344 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.005106 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.005337 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.058934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.059741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.108285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.108521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.110066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.110300 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.269136 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.269364 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.269592 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.269713 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.269802 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.269944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.270028 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.270166 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.270251 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.270383 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.270597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.270811 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.294684 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.294812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.294949 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.295072 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.295152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.295227 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.295342 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.295428 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.295565 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.295688 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.295770 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.295935 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.296021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.296119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.307436 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.307974 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.320669 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.320813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.321040 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.321259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.321344 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.321474 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.321558 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.321688 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.321772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.321901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.321985 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.322119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.322336 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.322552 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 30 00:34:11.339400 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.339611 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 30 00:34:11.339966 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.340192 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 30 00:34:11.340277 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.340403 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 30 00:34:11.340483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.340720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.340819 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.340927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 30 00:34:11.341009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.341198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.341279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.341459 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.341716 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.341971 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.351992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.352333 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.352490 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.352656 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.352801 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.352939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.353077 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.353213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.360277 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.362212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.363396 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.363802 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.364299 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.364775 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.365773 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.366857 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.367028 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.367169 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.367304 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.367450 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.367600 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.367735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.367868 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.368003 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.368186 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.368326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.368459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.368605 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.368923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.369376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.369867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.370212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.392247 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.393755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.394089 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.398511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.399491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.399863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.400194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.400374 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.400667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.406649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.406825 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.406966 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.407101 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.407408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.413587 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.413887 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.414176 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.414463 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.414715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.414817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.415010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.415093 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.415284 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.415557 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.420999 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.421298 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.421596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.421881 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.422168 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.422459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.422732 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.423012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.432129 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.432400 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.436258 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.439377 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.439474 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.439661 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.439744 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.444278 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.444897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.447525 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.447827 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.448034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.451273 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.451567 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.451851 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.452064 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.452172 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.452416 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.452703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.455205 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.458443 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.458777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.459291 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.459594 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.459887 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.460195 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.469141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.469438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.469705 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.469967 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.476458 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.476752 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.477023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.477289 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.477886 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.479429 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.481019 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.481290 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.483969 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.484170 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.484259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.484545 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.484837 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.485238 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.485724 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.487262 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.487552 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.490785 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.491067 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.491349 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.491633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.492022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.497573 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.497866 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.498152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.498436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.498823 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.499104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.499381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.503131 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.503430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.503713 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.503997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.504415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.504709 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.504990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.515201 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.518120 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.520538 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.521028 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.523201 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.523645 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.524705 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.524989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.525382 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.526620 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.528059 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.529558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.529850 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.530249 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.531696 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.533455 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.536333 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.537280 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.537466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.538078 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.538368 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.538944 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.541868 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.542265 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.542554 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.542838 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.546161 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.546448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.546724 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.550641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.550933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.551222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.555188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.555597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.555986 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.556414 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.556738 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.557014 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.557291 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.557585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.557861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.558131 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.558402 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.558662 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.558935 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.559261 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.559800 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 30 00:34:11.560173 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.560406 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.560686 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.560962 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.561236 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.561511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.561799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.562075 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.562362 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.562703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.562979 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.563245 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.563516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 30 00:34:11.594500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b370f9f9c0 pid: 88200 (default app) Oct 30 00:34:11.597519 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b370f9f9c0 pid: 88201 (default app) Oct 30 00:34:11.597909 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040809de-f1ab-4dd0-ad18-636782c2e98d None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:11.598198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040809de-f1ab-4dd0-ad18-636782c2e98d None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:11.598473 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040809de-f1ab-4dd0-ad18-636782c2e98d None None] PATH_INFO: `/v3` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:11.603836 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 791 bytes} [Sun Oct 30 00:34:11 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:18.507771 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e0ee9-f469-41cc-bb2a-dc1a4b9a6b70 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:18.508361 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e0ee9-f469-41cc-bb2a-dc1a4b9a6b70 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:18.508361 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e0ee9-f469-41cc-bb2a-dc1a4b9a6b70 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:18.512723 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1/2] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:34:18.518405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5acf4c-ea05-408f-b939-af59a664604b None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:18.519003 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5acf4c-ea05-408f-b939-af59a664604b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:18.519003 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5acf4c-ea05-408f-b939-af59a664604b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:18.591503 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: INFO dbcounter [None req-9a5acf4c-ea05-408f-b939-af59a664604b None None] Registered counter for database keystone Oct 30 00:34:18.611345 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG oslo_db.sqlalchemy.engines [None req-9a5acf4c-ea05-408f-b939-af59a664604b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88200) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:34:18.617931 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writer thread running {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:34:18.802438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG passlib.handlers.bcrypt [None req-9a5acf4c-ea05-408f-b939-af59a664604b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88200) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 30 00:34:18.802926 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG passlib.handlers.bcrypt [None req-9a5acf4c-ea05-408f-b939-af59a664604b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88200) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 30 00:34:18.864180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-9a5acf4c-ea05-408f-b939-af59a664604b None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:18.866353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-9a5acf4c-ea05-408f-b939-af59a664604b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:19.003931 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 2/3] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 486 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 30 00:34:19.471478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e0ee9-f469-41cc-bb2a-dc1a4b9a6b70 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:19.542629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: INFO dbcounter [None req-880e0ee9-f469-41cc-bb2a-dc1a4b9a6b70 None None] Registered counter for database keystone Oct 30 00:34:19.565340 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG oslo_db.sqlalchemy.engines [None req-880e0ee9-f469-41cc-bb2a-dc1a4b9a6b70 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88201) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 30 00:34:19.572685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writer thread running {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 30 00:34:19.694365 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e0ee9-f469-41cc-bb2a-dc1a4b9a6b70 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:19.698754 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744acecf-f796-4cc3-9c61-1e0790205058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:19.699393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744acecf-f796-4cc3-9c61-1e0790205058 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:19.699673 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744acecf-f796-4cc3-9c61-1e0790205058 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:19.699945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744acecf-f796-4cc3-9c61-1e0790205058 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:19.712322 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744acecf-f796-4cc3-9c61-1e0790205058 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:19.764627 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: WARNING keystone.common.rbac_enforcer.enforcer [None req-744acecf-f796-4cc3-9c61-1e0790205058 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 30 00:34:19.764997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744acecf-f796-4cc3-9c61-1e0790205058 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:19.775708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 2/4] 10.222.0.36 () {58 vars in 1324 bytes} [Sun Oct 30 00:34:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 00:34:19.781731 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5acf4c-ea05-408f-b939-af59a664604b None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:19.799821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5acf4c-ea05-408f-b939-af59a664604b None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:19.805086 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4b9d1-350b-47c5-8ce7-3d9415650871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:19.806246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4b9d1-350b-47c5-8ce7-3d9415650871 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:19.806533 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4b9d1-350b-47c5-8ce7-3d9415650871 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:19.807866 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4b9d1-350b-47c5-8ce7-3d9415650871 admin admin] PATH_INFO: `/v3/projects/3c2609b991ac4a86bca190e144b13290` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:19.808505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4b9d1-350b-47c5-8ce7-3d9415650871 admin admin] RBAC: Authorizing `identity:get_project(project_id=3c2609b991ac4a86bca190e144b13290)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:19.861141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: WARNING keystone.common.rbac_enforcer.enforcer [None req-24d4b9d1-350b-47c5-8ce7-3d9415650871 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 30 00:34:19.861494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4b9d1-350b-47c5-8ce7-3d9415650871 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:19.867007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 3/5] 10.222.0.36 () {56 vars in 1207 bytes} [Sun Oct 30 00:34:19 2022] GET /identity/v3/projects/3c2609b991ac4a86bca190e144b13290 => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:22.049979 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a94eda-a124-453c-a852-6a4a5664c46b None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:22.050563 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a94eda-a124-453c-a852-6a4a5664c46b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:22.050563 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a94eda-a124-453c-a852-6a4a5664c46b None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:22.055356 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 3/6] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:22.061228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df8fc6-0ace-474d-82b3-64d1d5af8383 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:22.061729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df8fc6-0ace-474d-82b3-64d1d5af8383 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:22.061834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df8fc6-0ace-474d-82b3-64d1d5af8383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:22.082466 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-70df8fc6-0ace-474d-82b3-64d1d5af8383 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:22.084126 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-70df8fc6-0ace-474d-82b3-64d1d5af8383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:22.092245 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 4/7] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 30 00:34:22.362670 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a94eda-a124-453c-a852-6a4a5664c46b None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:22.381527 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a94eda-a124-453c-a852-6a4a5664c46b None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:22.385815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3edefab-65dd-4ee5-ab6f-c121b480a34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:22.386433 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edefab-65dd-4ee5-ab6f-c121b480a34b admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:22.386715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edefab-65dd-4ee5-ab6f-c121b480a34b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:22.386991 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edefab-65dd-4ee5-ab6f-c121b480a34b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:22.395424 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3edefab-65dd-4ee5-ab6f-c121b480a34b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:22.397438 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3edefab-65dd-4ee5-ab6f-c121b480a34b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:22.406684 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 4/8] 10.222.0.36 () {58 vars in 1324 bytes} [Sun Oct 30 00:34:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 00:34:22.413431 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df8fc6-0ace-474d-82b3-64d1d5af8383 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:22.422729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df8fc6-0ace-474d-82b3-64d1d5af8383 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:22.427142 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818b150-fe9a-47b8-be87-467e8a7eccee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:22.428905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818b150-fe9a-47b8-be87-467e8a7eccee admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:22.429197 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818b150-fe9a-47b8-be87-467e8a7eccee admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:22.429475 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818b150-fe9a-47b8-be87-467e8a7eccee admin admin] PATH_INFO: `/v3/users/245a44aa1d7f43fcad2e727557036aab` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:22.430978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818b150-fe9a-47b8-be87-467e8a7eccee admin admin] RBAC: Authorizing `identity:get_user(user_id=245a44aa1d7f43fcad2e727557036aab)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:22.432845 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818b150-fe9a-47b8-be87-467e8a7eccee admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:22.434394 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 5/9] 10.222.0.36 () {56 vars in 1198 bytes} [Sun Oct 30 00:34:22 2022] GET /identity/v3/users/245a44aa1d7f43fcad2e727557036aab => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:24.683866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108806b-c80a-413d-a42b-4311ea348236 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:24.684464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108806b-c80a-413d-a42b-4311ea348236 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:24.684464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108806b-c80a-413d-a42b-4311ea348236 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:24.685780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 5/10] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:24.691265 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00b18c-c27a-4647-9496-970c02fdede9 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:24.691728 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00b18c-c27a-4647-9496-970c02fdede9 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:24.691835 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00b18c-c27a-4647-9496-970c02fdede9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:24.711160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-7a00b18c-c27a-4647-9496-970c02fdede9 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:24.712388 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-7a00b18c-c27a-4647-9496-970c02fdede9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:24.748178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c322b7-4a1b-441a-bb50-c4c54929bfbc None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:24.748461 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c322b7-4a1b-441a-bb50-c4c54929bfbc None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:24.748735 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c322b7-4a1b-441a-bb50-c4c54929bfbc None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:24.750263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 6/11] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:34:24.755355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ed377-6d25-47db-8b8a-c44fdbb6ec56 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:24.755641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ed377-6d25-47db-8b8a-c44fdbb6ec56 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:24.756837 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ed377-6d25-47db-8b8a-c44fdbb6ec56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:24.799744 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 6/12] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:24.799999 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG passlib.handlers.bcrypt [None req-d79ed377-6d25-47db-8b8a-c44fdbb6ec56 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88201) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 30 00:34:24.800317 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG passlib.handlers.bcrypt [None req-d79ed377-6d25-47db-8b8a-c44fdbb6ec56 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88201) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 30 00:34:24.850940 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-d79ed377-6d25-47db-8b8a-c44fdbb6ec56 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:24.852583 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d79ed377-6d25-47db-8b8a-c44fdbb6ec56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:24.956845 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 7/13] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:34:25.071304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00b18c-c27a-4647-9496-970c02fdede9 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:25.099573 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:25.189748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00b18c-c27a-4647-9496-970c02fdede9 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:25.224870 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ed377-6d25-47db-8b8a-c44fdbb6ec56 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:25.228668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d607c-6f99-40ef-a70e-a146f81b371e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:25.229323 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d607c-6f99-40ef-a70e-a146f81b371e None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:25.229609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d607c-6f99-40ef-a70e-a146f81b371e None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:25.229888 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d607c-6f99-40ef-a70e-a146f81b371e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:25.242006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d607c-6f99-40ef-a70e-a146f81b371e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:34:25.251064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-342d607c-6f99-40ef-a70e-a146f81b371e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:34:25.251064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:25.251064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:25.251064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:25.251064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:25.251064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:25.251064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:25.251064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:25.251064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:25.251064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:25.251064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:25.251064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:25.251064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:34:25.251064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:34:25.251064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:34:25.251064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:34:25.252228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:34:25.252228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:34:25.252228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:34:25.252228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:34:25.252228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:25.252228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:25.252228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:25.252228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:25.252228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:25.252228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:25.252228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:25.252228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:25.252228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:25.252228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:25.252228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:25.252228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:25.253198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:25.253198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:25.253198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:25.253198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:25.253198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:34:25.253198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:34:25.253198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:34:25.253198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:34:25.253198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:34:25.253198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:34:25.253198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:34:25.253198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:34:25.253198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 7/14] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:34:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:34:25.258738 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d607c-6f99-40ef-a70e-a146f81b371e None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:25.332845 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ed377-6d25-47db-8b8a-c44fdbb6ec56 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:25.335997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:25.354487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:25.372574 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244b1b7-63f2-4e23-bf72-379008194e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:25.373454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244b1b7-63f2-4e23-bf72-379008194e07 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:25.373735 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244b1b7-63f2-4e23-bf72-379008194e07 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:25.374007 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244b1b7-63f2-4e23-bf72-379008194e07 None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:25.374669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d607c-6f99-40ef-a70e-a146f81b371e None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:25.374989 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244b1b7-63f2-4e23-bf72-379008194e07 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:25.376448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244b1b7-63f2-4e23-bf72-379008194e07 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:25.388713 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.sql.core [None req-9244b1b7-63f2-4e23-bf72-379008194e07 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 30 00:34:25.388713 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 30 00:34:25.388713 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [parameters: {'id': '7674d09c84df4a318794e9c9dbac2ab6', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 30 00:34:25.388713 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88201) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 30 00:34:25.407703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-9244b1b7-63f2-4e23-bf72-379008194e07 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 30 00:34:25.407703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:25.407703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:34:25.407703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:34:25.407703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:34:25.407703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:34:25.407703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:34:25.407703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:34:25.407703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:34:25.407703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:34:25.407703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:34:25.407703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:34:25.407703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:34:25.407703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application result.read() Oct 30 00:34:25.409072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:34:25.409072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:34:25.409072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:34:25.409072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:34:25.409072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:34:25.409072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:34:25.409072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:34:25.409072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:34:25.409072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 30 00:34:25.409072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:25.409072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 30 00:34:25.409072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:25.409072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:25.409072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 30 00:34:25.409072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 30 00:34:25.409072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 30 00:34:25.409072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 30 00:34:25.409072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:34:25.409985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:34:25.409985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 30 00:34:25.409985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application yield resource Oct 30 00:34:25.409985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:34:25.409985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:34:25.409985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 30 00:34:25.409985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.session.rollback() Oct 30 00:34:25.409985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 30 00:34:25.409985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.force_reraise() Oct 30 00:34:25.409985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 30 00:34:25.409985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise self.value Oct 30 00:34:25.409985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 30 00:34:25.409985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 30 00:34:25.409985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 30 00:34:25.409985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application session.commit() Oct 30 00:34:25.409985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 30 00:34:25.409985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 30 00:34:25.410878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 30 00:34:25.410878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._prepare_impl() Oct 30 00:34:25.410878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 30 00:34:25.410878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.session.flush() Oct 30 00:34:25.410878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 30 00:34:25.410878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._flush(objects) Oct 30 00:34:25.410878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 30 00:34:25.410878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 30 00:34:25.410878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 30 00:34:25.410878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application compat.raise_( Oct 30 00:34:25.410878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:34:25.410878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception Oct 30 00:34:25.410878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 30 00:34:25.410878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application flush_context.execute() Oct 30 00:34:25.410878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 30 00:34:25.410878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rec.execute(self) Oct 30 00:34:25.410878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 30 00:34:25.411793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 30 00:34:25.411793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 30 00:34:25.411793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 30 00:34:25.411793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 30 00:34:25.411793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 30 00:34:25.411793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 30 00:34:25.411793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 30 00:34:25.411793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 30 00:34:25.411793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 30 00:34:25.411793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 30 00:34:25.411793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 30 00:34:25.411793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 30 00:34:25.411793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 30 00:34:25.411793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 30 00:34:25.411793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 30 00:34:25.412596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:34:25.412596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception Oct 30 00:34:25.412596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:34:25.412596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:34:25.412596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:34:25.412596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:34:25.412596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:34:25.412596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:34:25.412596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:34:25.412596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:34:25.412596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:34:25.412596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:34:25.412596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:34:25.412596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application result.read() Oct 30 00:34:25.412596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:34:25.412596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:34:25.412596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:34:25.413401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:34:25.413401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:34:25.413401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:34:25.413401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:34:25.413401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:34:25.413401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 30 00:34:25.413401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 30 00:34:25.413401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [parameters: {'id': '7674d09c84df4a318794e9c9dbac2ab6', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 30 00:34:25.413401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 30 00:34:25.413401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:25.413401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:34:25.413401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:25.413401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:25.413401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 30 00:34:25.413401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 30 00:34:25.414139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 30 00:34:25.414139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 30 00:34:25.414139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 30 00:34:25.414139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:25.414139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:34:25.414139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:25.414139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:25.414139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:25.414139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:25.414139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:25.414139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:25.414139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:25.414139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:25.414139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:25.414139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:25.414139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:25.414139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:25.414977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 30 00:34:25.414977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 30 00:34:25.414977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:25.414977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:25.414977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 30 00:34:25.414977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 30 00:34:25.414977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:25.414977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:25.414977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 30 00:34:25.414977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 30 00:34:25.414977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 30 00:34:25.414977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:25.414977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 8/15] 10.222.0.36 () {60 vars in 1137 bytes} [Sun Oct 30 00:34:25 2022] POST /identity/v3/domains => generated 190 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:34:25.415621 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366cf33-09cb-40ec-a2a0-bcca78d30557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:25.415621 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366cf33-09cb-40ec-a2a0-bcca78d30557 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:25.415621 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366cf33-09cb-40ec-a2a0-bcca78d30557 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:25.415621 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366cf33-09cb-40ec-a2a0-bcca78d30557 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:25.415858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244b1b7-63f2-4e23-bf72-379008194e07 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:25.415920 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f366cf33-09cb-40ec-a2a0-bcca78d30557 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:25.416156 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f366cf33-09cb-40ec-a2a0-bcca78d30557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:25.417695 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f366cf33-09cb-40ec-a2a0-bcca78d30557 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:25.431039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 8/16] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:34:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:34:25.436817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366cf33-09cb-40ec-a2a0-bcca78d30557 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:25.534013 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244b1b7-63f2-4e23-bf72-379008194e07 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:25.547673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366cf33-09cb-40ec-a2a0-bcca78d30557 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:25.568334 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ded353-bf08-4fff-936d-89796e20acec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:25.569749 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ded353-bf08-4fff-936d-89796e20acec None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:25.570083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ded353-bf08-4fff-936d-89796e20acec None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:25.570286 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ded353-bf08-4fff-936d-89796e20acec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:25.585385 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10966b5-d1af-4ad5-b071-8329ac893a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:25.586037 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10966b5-d1af-4ad5-b071-8329ac893a15 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:25.586311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10966b5-d1af-4ad5-b071-8329ac893a15 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:25.586587 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10966b5-d1af-4ad5-b071-8329ac893a15 None admin] PATH_INFO: `/v3/users/245a44aa1d7f43fcad2e727557036aab` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:25.587767 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10966b5-d1af-4ad5-b071-8329ac893a15 None admin] RBAC: Authorizing `identity:get_user(user_id=245a44aa1d7f43fcad2e727557036aab)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:25.589290 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10966b5-d1af-4ad5-b071-8329ac893a15 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:25.590183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ded353-bf08-4fff-936d-89796e20acec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:25.590911 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 9/17] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:34:25 2022] GET /identity/v3/users/245a44aa1d7f43fcad2e727557036aab => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:34:25.591676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ded353-bf08-4fff-936d-89796e20acec None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:25.596671 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10966b5-d1af-4ad5-b071-8329ac893a15 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:25.608852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-27ded353-bf08-4fff-936d-89796e20acec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:34:25.608852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:25.608852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:25.608852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:25.608852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:25.608852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:25.608852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:25.608852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:25.608852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:25.608852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:25.608852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:25.608852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:25.608852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:34:25.608852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:34:25.608852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:34:25.608852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:34:25.609858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:25.609858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:25.609858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:25.609858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:25.609858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:25.609858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:25.609858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:25.609858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:25.609858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:25.609858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:25.609858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:25.609858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:25.609858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:25.609858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:25.609858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:25.609858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:25.609858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:34:25.610674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:34:25.610674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:34:25.610674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:25.610674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 9/18] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:34:25.612038 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:25.616673 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ded353-bf08-4fff-936d-89796e20acec None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:25.645175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:25.705715 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10966b5-d1af-4ad5-b071-8329ac893a15 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:25.729294 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ded353-bf08-4fff-936d-89796e20acec None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:25.753952 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82ad26-9177-4217-8e99-db8e3fc39220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:25.754606 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82ad26-9177-4217-8e99-db8e3fc39220 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:25.754889 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82ad26-9177-4217-8e99-db8e3fc39220 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:25.755166 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82ad26-9177-4217-8e99-db8e3fc39220 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:25.756546 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82ad26-9177-4217-8e99-db8e3fc39220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:25.758057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82ad26-9177-4217-8e99-db8e3fc39220 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:25.759655 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 10/19] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:34:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:34:25.765373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82ad26-9177-4217-8e99-db8e3fc39220 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:25.767183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf76dd-b259-48d5-8a03-cedbe4ef6fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:25.767871 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf76dd-b259-48d5-8a03-cedbe4ef6fbb None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:25.768238 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf76dd-b259-48d5-8a03-cedbe4ef6fbb None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:25.768476 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf76dd-b259-48d5-8a03-cedbe4ef6fbb None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:25.769131 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf76dd-b259-48d5-8a03-cedbe4ef6fbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:25.769435 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf76dd-b259-48d5-8a03-cedbe4ef6fbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:25.770843 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf76dd-b259-48d5-8a03-cedbe4ef6fbb None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:25.790278 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 10/20] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:34:25.855932 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:25.879333 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82ad26-9177-4217-8e99-db8e3fc39220 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:25.919305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43446a0-71e9-47f5-b6dd-968bcbc468a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:25.919999 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43446a0-71e9-47f5-b6dd-968bcbc468a3 None admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:25.920312 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43446a0-71e9-47f5-b6dd-968bcbc468a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:25.920592 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43446a0-71e9-47f5-b6dd-968bcbc468a3 None admin] PATH_INFO: `/v3/domains/default/users/245a44aa1d7f43fcad2e727557036aab/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:25.922661 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43446a0-71e9-47f5-b6dd-968bcbc468a3 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=245a44aa1d7f43fcad2e727557036aab, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:25.924229 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43446a0-71e9-47f5-b6dd-968bcbc468a3 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:26.045593 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 11/21] 10.222.0.36 () {58 vars in 1361 bytes} [Sun Oct 30 00:34:25 2022] PUT /identity/v3/domains/default/users/245a44aa1d7f43fcad2e727557036aab/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:34:28.586117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ae638-758b-470b-ad2f-11198328e605 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:28.586807 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ae638-758b-470b-ad2f-11198328e605 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:28.586807 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ae638-758b-470b-ad2f-11198328e605 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:28.588319 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 11/22] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:28.597079 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d706bdc-9b77-45cd-8fe4-93c26372ade1 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:28.597600 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d706bdc-9b77-45cd-8fe4-93c26372ade1 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:28.597671 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d706bdc-9b77-45cd-8fe4-93c26372ade1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:28.632439 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-7d706bdc-9b77-45cd-8fe4-93c26372ade1 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:28.634094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-7d706bdc-9b77-45cd-8fe4-93c26372ade1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:28.735433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 12/23] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:34:28.739036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89108940-17d6-4a4e-b3d6-6d80d0040362 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:28.739328 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89108940-17d6-4a4e-b3d6-6d80d0040362 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:28.739602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89108940-17d6-4a4e-b3d6-6d80d0040362 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:28.741023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 12/24] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:28.746875 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c733262-7c7a-4eb6-9314-f66a8be9c0e3 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:28.747170 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c733262-7c7a-4eb6-9314-f66a8be9c0e3 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:28.747453 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c733262-7c7a-4eb6-9314-f66a8be9c0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:28.777271 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-4c733262-7c7a-4eb6-9314-f66a8be9c0e3 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:28.778524 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4c733262-7c7a-4eb6-9314-f66a8be9c0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:28.877637 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:28.883352 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 13/25] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:34:29.087433 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1bb92-d9e0-45b0-8975-eb2ddbfd513e None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:29.087986 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1bb92-d9e0-45b0-8975-eb2ddbfd513e None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:29.088426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1bb92-d9e0-45b0-8975-eb2ddbfd513e None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:29.089920 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 13/26] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:29.095622 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec24abe-054a-4127-aca2-72ed6c69c393 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:29.095915 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec24abe-054a-4127-aca2-72ed6c69c393 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:29.096231 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec24abe-054a-4127-aca2-72ed6c69c393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:29.130084 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-bec24abe-054a-4127-aca2-72ed6c69c393 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:29.131299 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-bec24abe-054a-4127-aca2-72ed6c69c393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:29.148480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1bb92-d9e0-45b0-8975-eb2ddbfd513e None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:29.241887 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 14/27] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:29.243880 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d4597-23a1-4e6c-8264-14167db99a77 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:29.244344 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d4597-23a1-4e6c-8264-14167db99a77 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:29.244641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d4597-23a1-4e6c-8264-14167db99a77 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:29.248995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 15/28] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:29.255411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a532d-b6f4-4aaa-b3f8-9a1cdbce27d5 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:29.256627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a532d-b6f4-4aaa-b3f8-9a1cdbce27d5 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:29.256627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a532d-b6f4-4aaa-b3f8-9a1cdbce27d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:29.277553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-655a532d-b6f4-4aaa-b3f8-9a1cdbce27d5 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:29.278772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-655a532d-b6f4-4aaa-b3f8-9a1cdbce27d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:29.340110 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:29.385066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1bb92-d9e0-45b0-8975-eb2ddbfd513e None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:29.388722 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 16/29] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:34:29.424418 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631827b-1359-4445-9dfa-a45feedf732b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:29.425114 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631827b-1359-4445-9dfa-a45feedf732b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:29.425404 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631827b-1359-4445-9dfa-a45feedf732b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:29.425683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631827b-1359-4445-9dfa-a45feedf732b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:29.437790 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631827b-1359-4445-9dfa-a45feedf732b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:29.439299 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631827b-1359-4445-9dfa-a45feedf732b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:29.455883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-9631827b-1359-4445-9dfa-a45feedf732b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:34:29.455883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:29.455883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:29.455883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:29.455883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:29.455883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:29.455883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:29.455883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:29.455883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:29.455883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:29.455883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:29.455883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:29.455883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:34:29.455883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:34:29.455883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:34:29.455883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:34:29.456863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:29.456863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:29.456863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:29.456863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:29.456863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:29.456863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:29.456863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:29.456863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:29.456863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:29.456863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:29.456863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:29.456863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:29.456863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:29.456863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:29.456863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:29.456863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:29.456863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:34:29.457669 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:34:29.457669 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:34:29.457669 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:29.457669 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 14/30] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:34:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:34:29.463301 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a532d-b6f4-4aaa-b3f8-9a1cdbce27d5 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:29.482978 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631827b-1359-4445-9dfa-a45feedf732b None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:29.582149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:29.600402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631827b-1359-4445-9dfa-a45feedf732b None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:29.655501 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:29.656904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a532d-b6f4-4aaa-b3f8-9a1cdbce27d5 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:29.664992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d203d-f3cc-492d-a230-9b40abd82cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:29.679873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d203d-f3cc-492d-a230-9b40abd82cd2 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:29.680195 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d203d-f3cc-492d-a230-9b40abd82cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:29.680459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d203d-f3cc-492d-a230-9b40abd82cd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:29.702279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d203d-f3cc-492d-a230-9b40abd82cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:34:29.709813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-848d203d-f3cc-492d-a230-9b40abd82cd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:34:29.709813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:29.709813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:29.709813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:29.709813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:29.709813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:29.709813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:29.709813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:29.709813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:29.709813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:29.709813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:29.709813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:29.709813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:34:29.709813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:34:29.709813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:34:29.709813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:34:29.710768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:34:29.710768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:34:29.710768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:34:29.710768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:34:29.710768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:29.710768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:29.710768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:29.710768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:29.710768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:29.710768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:29.710768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:29.710768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:29.710768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:29.710768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:29.710768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:29.710768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:29.712204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:29.712204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:29.712204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:29.712204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:29.712204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:34:29.712204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:34:29.712204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:34:29.712204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:34:29.712204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:34:29.712204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:34:29.712204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:34:29.712204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:29.712204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 15/31] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:34:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:34:29.712204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d203d-f3cc-492d-a230-9b40abd82cd2 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:29.718882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80ee88-066b-40d7-afa9-89048b53ad0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:29.719527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80ee88-066b-40d7-afa9-89048b53ad0e None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:29.719816 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80ee88-066b-40d7-afa9-89048b53ad0e None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:29.720124 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80ee88-066b-40d7-afa9-89048b53ad0e None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:29.720935 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80ee88-066b-40d7-afa9-89048b53ad0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:29.721247 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80ee88-066b-40d7-afa9-89048b53ad0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:29.722819 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80ee88-066b-40d7-afa9-89048b53ad0e None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:29.759082 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 17/32] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:29.765100 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80ee88-066b-40d7-afa9-89048b53ad0e None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:29.878395 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d203d-f3cc-492d-a230-9b40abd82cd2 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:29.923485 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80ee88-066b-40d7-afa9-89048b53ad0e None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:29.931595 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6aebe-6107-4856-9e0e-b72cf6c13f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:29.932275 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6aebe-6107-4856-9e0e-b72cf6c13f71 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:29.932546 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6aebe-6107-4856-9e0e-b72cf6c13f71 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:29.932823 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6aebe-6107-4856-9e0e-b72cf6c13f71 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:29.933510 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6aebe-6107-4856-9e0e-b72cf6c13f71 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:29.934903 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6aebe-6107-4856-9e0e-b72cf6c13f71 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:29.954973 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:29.975689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a240bae-2d44-4c68-ad89-acfc7dfefd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:29.976830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a240bae-2d44-4c68-ad89-acfc7dfefd93 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:29.977355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a240bae-2d44-4c68-ad89-acfc7dfefd93 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:29.977911 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a240bae-2d44-4c68-ad89-acfc7dfefd93 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:29.978901 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a240bae-2d44-4c68-ad89-acfc7dfefd93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:29.979458 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a240bae-2d44-4c68-ad89-acfc7dfefd93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:29.981231 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a240bae-2d44-4c68-ad89-acfc7dfefd93 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:29.992993 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 18/33] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:34:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:29.994338 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a240bae-2d44-4c68-ad89-acfc7dfefd93 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:30.084532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 16/34] 10.222.0.36 () {60 vars in 1131 bytes} [Sun Oct 30 00:34:29 2022] POST /identity/v3/roles => generated 219 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:34:30.085493 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6aebe-6107-4856-9e0e-b72cf6c13f71 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:30.150831 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a240bae-2d44-4c68-ad89-acfc7dfefd93 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:30.232854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:30.300545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6aebe-6107-4856-9e0e-b72cf6c13f71 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:30.352352 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f81ee2-077d-41bd-9724-a407ddf5f6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:30.353100 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f81ee2-077d-41bd-9724-a407ddf5f6cf None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:30.353501 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f81ee2-077d-41bd-9724-a407ddf5f6cf None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:30.353657 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f81ee2-077d-41bd-9724-a407ddf5f6cf None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:30.354377 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f81ee2-077d-41bd-9724-a407ddf5f6cf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:30.356135 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f81ee2-077d-41bd-9724-a407ddf5f6cf None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:30.362071 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471f6a5-d684-49ac-86d7-9dc653553a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:30.365910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471f6a5-d684-49ac-86d7-9dc653553a59 None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:30.365910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471f6a5-d684-49ac-86d7-9dc653553a59 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:30.365910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471f6a5-d684-49ac-86d7-9dc653553a59 None admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:30.365910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471f6a5-d684-49ac-86d7-9dc653553a59 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:30.366837 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471f6a5-d684-49ac-86d7-9dc653553a59 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:30.375650 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 17/35] 10.222.0.36 () {60 vars in 1131 bytes} [Sun Oct 30 00:34:30 2022] POST /identity/v3/roles => generated 225 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:34:30.376852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f81ee2-077d-41bd-9724-a407ddf5f6cf None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:30.411456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 19/36] 10.222.0.36 () {60 vars in 1141 bytes} [Sun Oct 30 00:34:29 2022] POST /identity/v3/projects => generated 301 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:34:30.411833 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f61c8-2090-4fdb-a18c-418278339edd None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:30.412240 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f61c8-2090-4fdb-a18c-418278339edd None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:30.412642 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f61c8-2090-4fdb-a18c-418278339edd None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:30.414694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 20/37] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:30.422249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf63d75-b0fa-4528-bce6-8ca5a03ca533 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:30.422592 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf63d75-b0fa-4528-bce6-8ca5a03ca533 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:30.422798 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf63d75-b0fa-4528-bce6-8ca5a03ca533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:30.442978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-adf63d75-b0fa-4528-bce6-8ca5a03ca533 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:30.444823 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-adf63d75-b0fa-4528-bce6-8ca5a03ca533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:30.500801 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:30.531387 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f81ee2-077d-41bd-9724-a407ddf5f6cf None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:30.534030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 21/38] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:30.540793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8ee72-0814-48c4-9a94-1c34403ba6d1 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:30.541248 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8ee72-0814-48c4-9a94-1c34403ba6d1 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:30.541754 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8ee72-0814-48c4-9a94-1c34403ba6d1 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:30.543683 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 22/39] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:30.549887 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec649b-5634-4b7a-8470-cea40622ab8b None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:30.550280 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec649b-5634-4b7a-8470-cea40622ab8b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:30.550718 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec649b-5634-4b7a-8470-cea40622ab8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:30.550972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:30.586753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-8dec649b-5634-4b7a-8470-cea40622ab8b None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:30.588522 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835b4b3-ec5a-4b54-a847-56c88ccde8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:30.589488 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835b4b3-ec5a-4b54-a847-56c88ccde8ac None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:30.589922 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835b4b3-ec5a-4b54-a847-56c88ccde8ac None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:30.590386 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835b4b3-ec5a-4b54-a847-56c88ccde8ac None admin] PATH_INFO: `/v3/users/245a44aa1d7f43fcad2e727557036aab` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:30.592288 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-8dec649b-5634-4b7a-8470-cea40622ab8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:30.594268 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835b4b3-ec5a-4b54-a847-56c88ccde8ac None admin] RBAC: Authorizing `identity:get_user(user_id=245a44aa1d7f43fcad2e727557036aab)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:30.597091 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835b4b3-ec5a-4b54-a847-56c88ccde8ac None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:30.599212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 18/40] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:34:30 2022] GET /identity/v3/users/245a44aa1d7f43fcad2e727557036aab => generated 247 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:30.606403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835b4b3-ec5a-4b54-a847-56c88ccde8ac None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:30.722968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 23/41] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:30.751437 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835b4b3-ec5a-4b54-a847-56c88ccde8ac None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:30.793322 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d98788-ad01-411d-bb4f-37d08155ea23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:30.794228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d98788-ad01-411d-bb4f-37d08155ea23 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:30.794656 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d98788-ad01-411d-bb4f-37d08155ea23 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:30.795088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d98788-ad01-411d-bb4f-37d08155ea23 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:30.796365 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d98788-ad01-411d-bb4f-37d08155ea23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:30.797886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d98788-ad01-411d-bb4f-37d08155ea23 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:30.799729 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 19/42] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:34:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:30.807845 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec649b-5634-4b7a-8470-cea40622ab8b None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:30.882979 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d98788-ad01-411d-bb4f-37d08155ea23 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:30.920213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:30.929755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec649b-5634-4b7a-8470-cea40622ab8b None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:30.941531 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:30.979185 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5eed4d-6b30-4186-858f-6270c82b3850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:30.979185 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5eed4d-6b30-4186-858f-6270c82b3850 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:30.979185 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5eed4d-6b30-4186-858f-6270c82b3850 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:30.979185 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5eed4d-6b30-4186-858f-6270c82b3850 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:30.979784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5eed4d-6b30-4186-858f-6270c82b3850 None admin] RBAC: Adding query filter params (role.id=980241d45fdb435d9f32c434057c71e4, scope.domain.id=default, user.id=245a44aa1d7f43fcad2e727557036aab) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:30.979986 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5eed4d-6b30-4186-858f-6270c82b3850 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:30.981523 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5eed4d-6b30-4186-858f-6270c82b3850 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:30.989159 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 24/43] 10.222.0.36 () {56 vars in 1322 bytes} [Sun Oct 30 00:34:30 2022] GET /identity/v3/role_assignments?user.id=245a44aa1d7f43fcad2e727557036aab&scope.domain.id=default&role.id=980241d45fdb435d9f32c434057c71e4 => generated 530 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:31.003289 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5eed4d-6b30-4186-858f-6270c82b3850 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:31.016160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d98788-ad01-411d-bb4f-37d08155ea23 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:31.058822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb261e-28fe-4352-8f70-e55ec423fe53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:31.059466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb261e-28fe-4352-8f70-e55ec423fe53 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:31.059748 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb261e-28fe-4352-8f70-e55ec423fe53 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:31.060024 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb261e-28fe-4352-8f70-e55ec423fe53 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:31.060756 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb261e-28fe-4352-8f70-e55ec423fe53 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:31.062308 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb261e-28fe-4352-8f70-e55ec423fe53 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:31.078375 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 20/44] 10.222.0.36 () {60 vars in 1131 bytes} [Sun Oct 30 00:34:30 2022] POST /identity/v3/roles => generated 223 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:34:31.137230 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5eed4d-6b30-4186-858f-6270c82b3850 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:31.182156 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476fd74f-4a75-4c07-8e79-1824efd66df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:31.183209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fd74f-4a75-4c07-8e79-1824efd66df5 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:31.183664 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fd74f-4a75-4c07-8e79-1824efd66df5 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:31.184131 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fd74f-4a75-4c07-8e79-1824efd66df5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:31.185649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476fd74f-4a75-4c07-8e79-1824efd66df5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:31.187436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476fd74f-4a75-4c07-8e79-1824efd66df5 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:31.189406 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 25/45] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:34:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:34:31.195551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb261e-28fe-4352-8f70-e55ec423fe53 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:31.319759 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb261e-28fe-4352-8f70-e55ec423fe53 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:31.329980 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:31.360182 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd42086-d365-45c2-b59b-b31d792401b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:31.360819 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd42086-d365-45c2-b59b-b31d792401b4 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:31.361106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd42086-d365-45c2-b59b-b31d792401b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:31.361386 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd42086-d365-45c2-b59b-b31d792401b4 None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:31.362046 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd42086-d365-45c2-b59b-b31d792401b4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:31.363518 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd42086-d365-45c2-b59b-b31d792401b4 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:31.391751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 21/46] 10.222.0.36 () {60 vars in 1141 bytes} [Sun Oct 30 00:34:31 2022] POST /identity/v3/projects => generated 312 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:34:33.575235 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e785fe-3427-432e-9e53-69b023ce5000 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:33.575763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e785fe-3427-432e-9e53-69b023ce5000 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:33.575763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e785fe-3427-432e-9e53-69b023ce5000 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:33.577266 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 26/47] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:33.582691 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fd054-fe28-4386-9076-b9829aa75f48 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:33.583138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fd054-fe28-4386-9076-b9829aa75f48 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:33.583295 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fd054-fe28-4386-9076-b9829aa75f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:33.602592 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-9e0fd054-fe28-4386-9076-b9829aa75f48 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:33.604167 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-9e0fd054-fe28-4386-9076-b9829aa75f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:33.684477 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 22/48] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:33.960748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e785fe-3427-432e-9e53-69b023ce5000 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:33.988420 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:34.063835 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e785fe-3427-432e-9e53-69b023ce5000 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:34.101606 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf25e1-3528-4dff-8486-e53af25d3449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:34.102244 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf25e1-3528-4dff-8486-e53af25d3449 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:34.102530 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf25e1-3528-4dff-8486-e53af25d3449 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:34.102809 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf25e1-3528-4dff-8486-e53af25d3449 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:34.103942 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf25e1-3528-4dff-8486-e53af25d3449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:34.105448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf25e1-3528-4dff-8486-e53af25d3449 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:34.107061 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 27/49] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:34:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:34.112891 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fd054-fe28-4386-9076-b9829aa75f48 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:34.200932 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:34.211923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fd054-fe28-4386-9076-b9829aa75f48 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:34.246258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a26a3a-cd98-48f3-b066-e7c7b47abc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:34.246871 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a26a3a-cd98-48f3-b066-e7c7b47abc75 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:34.247161 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a26a3a-cd98-48f3-b066-e7c7b47abc75 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:34.247608 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a26a3a-cd98-48f3-b066-e7c7b47abc75 None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:34.248299 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a26a3a-cd98-48f3-b066-e7c7b47abc75 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:34.249772 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a26a3a-cd98-48f3-b066-e7c7b47abc75 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:34.276189 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 23/50] 10.222.0.36 () {60 vars in 1140 bytes} [Sun Oct 30 00:34:34 2022] POST /identity/v3/projects => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:34:36.495956 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb2a1f-3b74-41d6-b004-78125e1606d7 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:36.496550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb2a1f-3b74-41d6-b004-78125e1606d7 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:36.496550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb2a1f-3b74-41d6-b004-78125e1606d7 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:36.497895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 28/51] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:36.503449 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1e217-34b4-4dd4-abe3-595d9dd91af1 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:36.503905 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1e217-34b4-4dd4-abe3-595d9dd91af1 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:36.504017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1e217-34b4-4dd4-abe3-595d9dd91af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:36.522055 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-44d1e217-34b4-4dd4-abe3-595d9dd91af1 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:36.523262 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-44d1e217-34b4-4dd4-abe3-595d9dd91af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:36.596500 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 24/52] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:36.872320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb2a1f-3b74-41d6-b004-78125e1606d7 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:36.968217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:36.977205 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb2a1f-3b74-41d6-b004-78125e1606d7 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:37.013081 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22a675-bc86-45d4-9d71-549b7d6b10fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:37.013722 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22a675-bc86-45d4-9d71-549b7d6b10fc None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:37.014007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22a675-bc86-45d4-9d71-549b7d6b10fc None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:37.014275 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22a675-bc86-45d4-9d71-549b7d6b10fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:37.015408 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22a675-bc86-45d4-9d71-549b7d6b10fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:37.016895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22a675-bc86-45d4-9d71-549b7d6b10fc None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:37.018491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 29/53] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:34:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:37.024275 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1e217-34b4-4dd4-abe3-595d9dd91af1 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:37.095372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:37.132345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1e217-34b4-4dd4-abe3-595d9dd91af1 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:37.169606 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b517bf-0944-4115-9407-d016b4382e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:37.170226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b517bf-0944-4115-9407-d016b4382e6c None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:37.170506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b517bf-0944-4115-9407-d016b4382e6c None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:37.170776 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b517bf-0944-4115-9407-d016b4382e6c None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:37.171438 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b517bf-0944-4115-9407-d016b4382e6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:37.172917 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b517bf-0944-4115-9407-d016b4382e6c None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:37.201851 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 25/54] 10.222.0.36 () {60 vars in 1132 bytes} [Sun Oct 30 00:34:37 2022] POST /identity/v3/users => generated 275 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:34:40.661564 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acfa66-8a5e-42a9-8d6c-86e3bb7e5567 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:40.662995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acfa66-8a5e-42a9-8d6c-86e3bb7e5567 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:40.663425 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acfa66-8a5e-42a9-8d6c-86e3bb7e5567 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:40.665359 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 30/55] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:40.670842 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0200cdb-2f94-4619-b877-76b4cb83336a None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:40.671653 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0200cdb-2f94-4619-b877-76b4cb83336a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:40.672135 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0200cdb-2f94-4619-b877-76b4cb83336a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:40.820762 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-a0200cdb-2f94-4619-b877-76b4cb83336a None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:40.822209 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a0200cdb-2f94-4619-b877-76b4cb83336a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:40.937777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 26/56] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:41.067818 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a73b33-a6e9-4d6f-91e2-98ddcfa597cf None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:41.068367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a73b33-a6e9-4d6f-91e2-98ddcfa597cf None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:41.069307 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a73b33-a6e9-4d6f-91e2-98ddcfa597cf None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:41.071361 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 31/57] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:41.078250 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bc381-5a89-48fa-9fc3-8aba01106d85 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:41.078760 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bc381-5a89-48fa-9fc3-8aba01106d85 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:41.079178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bc381-5a89-48fa-9fc3-8aba01106d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:41.104153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d621a53-56bd-4058-b084-6ffcfcacc433 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:41.104791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d621a53-56bd-4058-b084-6ffcfcacc433 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:41.105319 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d621a53-56bd-4058-b084-6ffcfcacc433 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:41.106862 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 32/58] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:34:41.112717 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-255bc381-5a89-48fa-9fc3-8aba01106d85 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:41.113290 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc06842-f315-45dc-b3a7-e56f1285613b None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:41.113811 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc06842-f315-45dc-b3a7-e56f1285613b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:41.114144 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-255bc381-5a89-48fa-9fc3-8aba01106d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:41.114520 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc06842-f315-45dc-b3a7-e56f1285613b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:41.140653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-ebc06842-f315-45dc-b3a7-e56f1285613b None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:41.142405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-ebc06842-f315-45dc-b3a7-e56f1285613b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:41.220600 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:41.225154 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 33/59] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:34:41.227326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9c9db-e224-45ce-893d-67481f47f029 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:41.227825 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9c9db-e224-45ce-893d-67481f47f029 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:41.228327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9c9db-e224-45ce-893d-67481f47f029 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:41.230136 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 34/60] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:41.241146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 27/61] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:41.242649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81946858-402d-4289-b49f-7dec09b2fd40 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:41.242945 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81946858-402d-4289-b49f-7dec09b2fd40 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:41.243226 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81946858-402d-4289-b49f-7dec09b2fd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:41.273104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-81946858-402d-4289-b49f-7dec09b2fd40 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:41.274321 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-81946858-402d-4289-b49f-7dec09b2fd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:41.328347 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bc381-5a89-48fa-9fc3-8aba01106d85 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:41.382752 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 35/62] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:41.471256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bc381-5a89-48fa-9fc3-8aba01106d85 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:41.509212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48316fef-ada5-49d4-8b6c-6666de9359a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:41.510233 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48316fef-ada5-49d4-8b6c-6666de9359a6 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:41.510717 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48316fef-ada5-49d4-8b6c-6666de9359a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:41.511196 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48316fef-ada5-49d4-8b6c-6666de9359a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:41.523570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48316fef-ada5-49d4-8b6c-6666de9359a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:34:41.529112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-48316fef-ada5-49d4-8b6c-6666de9359a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:34:41.529112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:41.529112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:41.529112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:41.529112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:41.529112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:41.529112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:41.529112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:41.529112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:41.529112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:41.529112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:41.529112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:41.529112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:34:41.529112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:34:41.529112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:34:41.529112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:34:41.530052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:34:41.530052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:34:41.530052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:34:41.530052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:34:41.530052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:41.530052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:41.530052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:41.530052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:41.530052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:41.530052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:41.530052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:41.530052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:41.530052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:41.530052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:41.530052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:41.530052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:41.530854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:41.530854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:41.530854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:41.530854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:41.530854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:34:41.530854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:34:41.530854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:34:41.530854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:34:41.530854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:34:41.530854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:34:41.530854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:34:41.530854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:41.533451 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 28/63] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:34:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:34:41.541059 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81946858-402d-4289-b49f-7dec09b2fd40 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:41.582017 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:41.591545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48316fef-ada5-49d4-8b6c-6666de9359a6 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:41.660524 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:41.675461 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81946858-402d-4289-b49f-7dec09b2fd40 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:41.714671 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa1e83-3f89-429a-beda-1fe4bd13ab8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:41.715299 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa1e83-3f89-429a-beda-1fe4bd13ab8d None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:41.715587 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa1e83-3f89-429a-beda-1fe4bd13ab8d None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:41.715859 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa1e83-3f89-429a-beda-1fe4bd13ab8d None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:41.716578 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa1e83-3f89-429a-beda-1fe4bd13ab8d None admin] RBAC: Adding query filter params (name=member) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:41.716887 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa1e83-3f89-429a-beda-1fe4bd13ab8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:41.718305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa1e83-3f89-429a-beda-1fe4bd13ab8d None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:41.723406 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48316fef-ada5-49d4-8b6c-6666de9359a6 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:41.728129 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 36/64] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:34:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:41.728991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa1e83-3f89-429a-beda-1fe4bd13ab8d None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:41.771255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e39178-5cf3-4eae-9d89-584f76be6211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:41.772354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e39178-5cf3-4eae-9d89-584f76be6211 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:41.772564 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e39178-5cf3-4eae-9d89-584f76be6211 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:41.772826 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e39178-5cf3-4eae-9d89-584f76be6211 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:41.784736 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e39178-5cf3-4eae-9d89-584f76be6211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:34:41.790208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-b5e39178-5cf3-4eae-9d89-584f76be6211 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:34:41.790208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:41.790208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:41.790208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:41.790208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:41.790208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:41.790208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:41.790208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:41.790208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:41.790208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:41.790208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:41.790208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:41.790208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:34:41.790208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:34:41.790208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:34:41.790208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:34:41.791098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:34:41.791098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:34:41.791098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:34:41.791098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:34:41.791098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:41.791098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:41.791098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:41.791098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:41.791098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:41.791098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:41.791098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:41.791098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:41.791098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:41.791098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:41.791098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:41.791098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:41.797056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:41.797056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:41.797056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:41.797056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:41.797056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:34:41.797056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:34:41.797056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:34:41.797056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:34:41.797056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:34:41.797056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:34:41.797056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:34:41.797056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:41.797056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 29/65] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:34:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:34:41.797056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc954b-95a0-4234-9a62-849544413985 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:41.797741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc954b-95a0-4234-9a62-849544413985 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:41.797741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc954b-95a0-4234-9a62-849544413985 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:41.797741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 30/66] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:41.800175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc954b-95a0-4234-9a62-849544413985 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:41.836727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:41.855559 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa1e83-3f89-429a-beda-1fe4bd13ab8d None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:41.892022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea16f8c-9e8a-4c5a-bcb9-a0ce853b5c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:41.892707 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea16f8c-9e8a-4c5a-bcb9-a0ce853b5c20 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:41.892989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea16f8c-9e8a-4c5a-bcb9-a0ce853b5c20 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:41.893266 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea16f8c-9e8a-4c5a-bcb9-a0ce853b5c20 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:41.905105 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea16f8c-9e8a-4c5a-bcb9-a0ce853b5c20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:34:41.911391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-fea16f8c-9e8a-4c5a-bcb9-a0ce853b5c20 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:34:41.911391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:41.911391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:41.911391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:41.911391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:41.911391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:41.911391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:41.911391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:41.911391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:41.911391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:41.911391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:41.911391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:41.911391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:34:41.911391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:34:41.911391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:34:41.911391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:34:41.914043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:34:41.914043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:34:41.914043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:34:41.914043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:34:41.914043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:41.914043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:41.914043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:41.914043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:41.914043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:41.914043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:41.914043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:41.914043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:41.914043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:41.914043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:41.914043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:41.914043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:41.914883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:41.914883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:41.914883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:41.914883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:41.914883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:34:41.914883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:34:41.914883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:34:41.914883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:34:41.914883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:34:41.914883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:34:41.914883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:34:41.914883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:34:41.914883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 37/67] 10.222.0.36 () {56 vars in 1114 bytes} [Sun Oct 30 00:34:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:34:41.914883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea16f8c-9e8a-4c5a-bcb9-a0ce853b5c20 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:41.922375 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc954b-95a0-4234-9a62-849544413985 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:41.932910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:41.959950 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cee191-8be4-47b4-a2b8-b1abb359498d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:41.961032 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cee191-8be4-47b4-a2b8-b1abb359498d None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:41.961499 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cee191-8be4-47b4-a2b8-b1abb359498d None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:41.962688 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cee191-8be4-47b4-a2b8-b1abb359498d None admin] PATH_INFO: `/v3/users/9e8fef123072495eabcb616db3e670a6` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:41.982524 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cee191-8be4-47b4-a2b8-b1abb359498d None admin] RBAC: Authorizing `identity:get_user(user_id=9e8fef123072495eabcb616db3e670a6)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:41.983930 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cee191-8be4-47b4-a2b8-b1abb359498d None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:41.985672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 31/68] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:34:41 2022] GET /identity/v3/users/9e8fef123072495eabcb616db3e670a6 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:41.986844 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6e92b-f24f-4ce7-8c5a-ac5a8bb18edf None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:41.987131 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6e92b-f24f-4ce7-8c5a-ac5a8bb18edf None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:41.987400 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6e92b-f24f-4ce7-8c5a-ac5a8bb18edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:42.006054 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-80f6e92b-f24f-4ce7-8c5a-ac5a8bb18edf None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:42.007327 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-80f6e92b-f24f-4ce7-8c5a-ac5a8bb18edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:42.035679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea16f8c-9e8a-4c5a-bcb9-a0ce853b5c20 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:42.072209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce0580-19c6-4110-ac83-ee9a732ade7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:42.072841 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce0580-19c6-4110-ac83-ee9a732ade7a None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:42.073122 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce0580-19c6-4110-ac83-ee9a732ade7a None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:42.073396 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce0580-19c6-4110-ac83-ee9a732ade7a None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:42.074058 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce0580-19c6-4110-ac83-ee9a732ade7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:42.074360 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce0580-19c6-4110-ac83-ee9a732ade7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:42.075759 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce0580-19c6-4110-ac83-ee9a732ade7a None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:42.079635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 32/69] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:42.080533 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6e92b-f24f-4ce7-8c5a-ac5a8bb18edf None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:42.085125 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 38/70] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:34:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:42.086435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce0580-19c6-4110-ac83-ee9a732ade7a None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:42.097221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:42.208478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:42.212443 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce0580-19c6-4110-ac83-ee9a732ade7a None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:42.218526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6e92b-f24f-4ce7-8c5a-ac5a8bb18edf None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:42.251178 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409897a2-e34c-4664-99ce-3cb70eb214db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:42.252039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409897a2-e34c-4664-99ce-3cb70eb214db None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:42.252355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409897a2-e34c-4664-99ce-3cb70eb214db None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:42.252638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409897a2-e34c-4664-99ce-3cb70eb214db None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:42.253340 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409897a2-e34c-4664-99ce-3cb70eb214db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:42.253690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409897a2-e34c-4664-99ce-3cb70eb214db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:42.255104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409897a2-e34c-4664-99ce-3cb70eb214db None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:42.261619 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06146ff-f369-4cc5-9f80-da2edd0b5e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:42.262271 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06146ff-f369-4cc5-9f80-da2edd0b5e18 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:42.262548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06146ff-f369-4cc5-9f80-da2edd0b5e18 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:42.262817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06146ff-f369-4cc5-9f80-da2edd0b5e18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:42.265027 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 39/71] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:34:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:42.266125 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409897a2-e34c-4664-99ce-3cb70eb214db None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:42.279526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06146ff-f369-4cc5-9f80-da2edd0b5e18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:34:42.284758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-a06146ff-f369-4cc5-9f80-da2edd0b5e18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:34:42.284758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:42.284758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:42.284758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:42.284758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:42.284758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:42.284758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:42.284758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:42.284758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:42.284758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:42.284758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:42.284758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:42.284758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:34:42.284758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:34:42.284758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:34:42.284758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:34:42.285681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:34:42.285681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:34:42.285681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:34:42.285681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:34:42.285681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:42.285681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:42.285681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:42.285681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:42.285681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:42.285681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:42.285681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:42.285681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:42.285681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:42.285681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:42.285681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:42.285681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:42.286450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:42.286450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:42.286450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:42.286450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:42.286450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:34:42.286450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:34:42.286450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:34:42.286450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:34:42.286450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:34:42.286450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:34:42.286450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:34:42.286450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:42.286450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 33/72] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:34:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:34:42.287095 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06146ff-f369-4cc5-9f80-da2edd0b5e18 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:42.367480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:42.408542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409897a2-e34c-4664-99ce-3cb70eb214db None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:42.408792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06146ff-f369-4cc5-9f80-da2edd0b5e18 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:42.444113 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00941066-21b0-4eb5-a912-5e2079c577f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:42.444850 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00941066-21b0-4eb5-a912-5e2079c577f4 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:42.445142 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00941066-21b0-4eb5-a912-5e2079c577f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:42.445416 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00941066-21b0-4eb5-a912-5e2079c577f4 None admin] PATH_INFO: `/v3/projects/72d02777c6694301959a12c02452adf3` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:42.446582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00941066-21b0-4eb5-a912-5e2079c577f4 None admin] RBAC: Authorizing `identity:get_project(project_id=72d02777c6694301959a12c02452adf3)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:42.447311 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341db47-3c2a-4123-b292-a8cd69e1fbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:42.447958 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341db47-3c2a-4123-b292-a8cd69e1fbfb None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:42.448072 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00941066-21b0-4eb5-a912-5e2079c577f4 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:42.448279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341db47-3c2a-4123-b292-a8cd69e1fbfb None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:42.448560 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341db47-3c2a-4123-b292-a8cd69e1fbfb None admin] PATH_INFO: `/v3/users/245a44aa1d7f43fcad2e727557036aab` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:42.449730 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341db47-3c2a-4123-b292-a8cd69e1fbfb None admin] RBAC: Authorizing `identity:get_user(user_id=245a44aa1d7f43fcad2e727557036aab)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:42.449845 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 40/73] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:34:42 2022] GET /identity/v3/projects/72d02777c6694301959a12c02452adf3 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:42.450555 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00941066-21b0-4eb5-a912-5e2079c577f4 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:42.451189 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341db47-3c2a-4123-b292-a8cd69e1fbfb None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:42.452830 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 34/74] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:34:42 2022] GET /identity/v3/users/245a44aa1d7f43fcad2e727557036aab => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:42.457043 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341db47-3c2a-4123-b292-a8cd69e1fbfb None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:42.485289 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:42.576008 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00941066-21b0-4eb5-a912-5e2079c577f4 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:42.580568 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341db47-3c2a-4123-b292-a8cd69e1fbfb None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:42.611719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9d866-4d45-4cf6-b80f-98b696aac521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:42.612400 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9d866-4d45-4cf6-b80f-98b696aac521 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:42.612679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9d866-4d45-4cf6-b80f-98b696aac521 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:42.612960 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9d866-4d45-4cf6-b80f-98b696aac521 None admin] PATH_INFO: `/v3/users/9e8fef123072495eabcb616db3e670a6` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:42.614116 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9d866-4d45-4cf6-b80f-98b696aac521 None admin] RBAC: Authorizing `identity:get_user(user_id=9e8fef123072495eabcb616db3e670a6)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:42.616332 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9d866-4d45-4cf6-b80f-98b696aac521 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:42.618486 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 41/75] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:34:42 2022] GET /identity/v3/users/9e8fef123072495eabcb616db3e670a6 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:42.619268 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9d866-4d45-4cf6-b80f-98b696aac521 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:42.619450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93da0f-2717-482d-91d8-c654a49ff4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:42.620064 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93da0f-2717-482d-91d8-c654a49ff4f2 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:42.620349 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93da0f-2717-482d-91d8-c654a49ff4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:42.620629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93da0f-2717-482d-91d8-c654a49ff4f2 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:42.621304 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93da0f-2717-482d-91d8-c654a49ff4f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:42.621611 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93da0f-2717-482d-91d8-c654a49ff4f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:42.623015 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93da0f-2717-482d-91d8-c654a49ff4f2 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:42.624045 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:42.636585 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 35/76] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:34:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:42.637376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93da0f-2717-482d-91d8-c654a49ff4f2 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:42.726913 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:42.733816 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9d866-4d45-4cf6-b80f-98b696aac521 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:42.743611 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93da0f-2717-482d-91d8-c654a49ff4f2 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:42.772463 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a804f-5a96-4847-87e6-9af418923b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:42.773116 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a804f-5a96-4847-87e6-9af418923b1f None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:42.773473 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a804f-5a96-4847-87e6-9af418923b1f None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:42.773680 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a804f-5a96-4847-87e6-9af418923b1f None admin] PATH_INFO: `/v3/domains/default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:42.774876 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a804f-5a96-4847-87e6-9af418923b1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:42.776393 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a804f-5a96-4847-87e6-9af418923b1f None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:42.778029 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 42/77] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:34:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:42.778996 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a804f-5a96-4847-87e6-9af418923b1f None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:42.781150 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0bc32-c5df-405c-8cff-d4658d1a26c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:42.781840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0bc32-c5df-405c-8cff-d4658d1a26c6 None admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:42.782122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0bc32-c5df-405c-8cff-d4658d1a26c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:42.782400 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0bc32-c5df-405c-8cff-d4658d1a26c6 None admin] PATH_INFO: `/v3/projects/72d02777c6694301959a12c02452adf3/users/9e8fef123072495eabcb616db3e670a6/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:42.784732 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0bc32-c5df-405c-8cff-d4658d1a26c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=72d02777c6694301959a12c02452adf3, user_id=9e8fef123072495eabcb616db3e670a6, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:42.786214 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0bc32-c5df-405c-8cff-d4658d1a26c6 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:42.802773 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 36/78] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:34:42 2022] PUT /identity/v3/projects/72d02777c6694301959a12c02452adf3/users/9e8fef123072495eabcb616db3e670a6/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:34:42.804784 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0bc32-c5df-405c-8cff-d4658d1a26c6 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:42.901023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:42.947321 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a804f-5a96-4847-87e6-9af418923b1f None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:42.960633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0bc32-c5df-405c-8cff-d4658d1a26c6 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:42.992212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815875c5-9446-4247-bd4d-69783dd0ffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:42.993127 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815875c5-9446-4247-bd4d-69783dd0ffda None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:42.993479 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815875c5-9446-4247-bd4d-69783dd0ffda None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:42.993770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815875c5-9446-4247-bd4d-69783dd0ffda None admin] PATH_INFO: `/v3/projects/72d02777c6694301959a12c02452adf3` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:42.994975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815875c5-9446-4247-bd4d-69783dd0ffda None admin] RBAC: Authorizing `identity:get_project(project_id=72d02777c6694301959a12c02452adf3)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:42.996474 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815875c5-9446-4247-bd4d-69783dd0ffda None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:42.998149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 43/79] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:34:42 2022] GET /identity/v3/projects/72d02777c6694301959a12c02452adf3 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:42.998933 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815875c5-9446-4247-bd4d-69783dd0ffda None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:43.007594 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b53a9-641f-4be4-a4a5-a2dcf30f9ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:43.008345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b53a9-641f-4be4-a4a5-a2dcf30f9ce9 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:43.008628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b53a9-641f-4be4-a4a5-a2dcf30f9ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:43.008905 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b53a9-641f-4be4-a4a5-a2dcf30f9ce9 None admin] PATH_INFO: `/v3/projects/72d02777c6694301959a12c02452adf3` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:43.010179 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b53a9-641f-4be4-a4a5-a2dcf30f9ce9 None admin] RBAC: Authorizing `identity:get_project(project_id=72d02777c6694301959a12c02452adf3)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:43.011694 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b53a9-641f-4be4-a4a5-a2dcf30f9ce9 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:43.013624 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 37/80] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:34:42 2022] GET /identity/v3/projects/72d02777c6694301959a12c02452adf3 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:43.018729 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b53a9-641f-4be4-a4a5-a2dcf30f9ce9 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:43.033448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:43.121996 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815875c5-9446-4247-bd4d-69783dd0ffda None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:43.135181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b53a9-641f-4be4-a4a5-a2dcf30f9ce9 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:43.158376 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488d8e5-34bf-48c2-981e-3bba45eaeaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:43.159166 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488d8e5-34bf-48c2-981e-3bba45eaeaf8 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:43.159703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488d8e5-34bf-48c2-981e-3bba45eaeaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:43.159975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488d8e5-34bf-48c2-981e-3bba45eaeaf8 None admin] PATH_INFO: `/v3/users/9e8fef123072495eabcb616db3e670a6` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:43.161148 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9488d8e5-34bf-48c2-981e-3bba45eaeaf8 None admin] RBAC: Authorizing `identity:get_user(user_id=9e8fef123072495eabcb616db3e670a6)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:43.163188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9488d8e5-34bf-48c2-981e-3bba45eaeaf8 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:43.164930 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 44/81] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:34:42 2022] GET /identity/v3/users/9e8fef123072495eabcb616db3e670a6 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:43.165684 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488d8e5-34bf-48c2-981e-3bba45eaeaf8 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:43.177288 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:43.181264 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559390bc-e4eb-42ea-9949-ee6d031671f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:43.181895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559390bc-e4eb-42ea-9949-ee6d031671f2 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:43.182290 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559390bc-e4eb-42ea-9949-ee6d031671f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:43.182572 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559390bc-e4eb-42ea-9949-ee6d031671f2 None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:43.183470 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559390bc-e4eb-42ea-9949-ee6d031671f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:43.185039 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559390bc-e4eb-42ea-9949-ee6d031671f2 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:43.220185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 38/82] 10.222.0.36 () {60 vars in 1132 bytes} [Sun Oct 30 00:34:43 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:34:43.220185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559390bc-e4eb-42ea-9949-ee6d031671f2 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:43.310232 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488d8e5-34bf-48c2-981e-3bba45eaeaf8 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:43.337849 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:43.367650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb0ffe-b9d4-48a1-86ee-ff522dbf6b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:43.368738 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb0ffe-b9d4-48a1-86ee-ff522dbf6b5e None admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:43.369191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb0ffe-b9d4-48a1-86ee-ff522dbf6b5e None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:43.370481 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb0ffe-b9d4-48a1-86ee-ff522dbf6b5e None admin] PATH_INFO: `/v3/projects/72d02777c6694301959a12c02452adf3/users/245a44aa1d7f43fcad2e727557036aab/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:43.373601 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb0ffe-b9d4-48a1-86ee-ff522dbf6b5e None admin] RBAC: Authorizing `identity:create_grant(project_id=72d02777c6694301959a12c02452adf3, user_id=245a44aa1d7f43fcad2e727557036aab, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:43.376375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb0ffe-b9d4-48a1-86ee-ff522dbf6b5e None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:43.387110 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559390bc-e4eb-42ea-9949-ee6d031671f2 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:43.398761 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 45/83] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:34:43 2022] PUT /identity/v3/projects/72d02777c6694301959a12c02452adf3/users/245a44aa1d7f43fcad2e727557036aab/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:34:43.400779 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb0ffe-b9d4-48a1-86ee-ff522dbf6b5e None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:43.453605 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4fe63-eea1-41da-83e4-5dcd71b1c657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:43.455444 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4fe63-eea1-41da-83e4-5dcd71b1c657 None admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:43.461122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4fe63-eea1-41da-83e4-5dcd71b1c657 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:43.461392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4fe63-eea1-41da-83e4-5dcd71b1c657 None admin] PATH_INFO: `/v3/projects/72d02777c6694301959a12c02452adf3/users/9e8fef123072495eabcb616db3e670a6/roles/af3529d63bae4650be7b8dc67b3ed527` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:43.463862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4fe63-eea1-41da-83e4-5dcd71b1c657 None admin] RBAC: Authorizing `identity:create_grant(project_id=72d02777c6694301959a12c02452adf3, user_id=9e8fef123072495eabcb616db3e670a6, role_id=af3529d63bae4650be7b8dc67b3ed527)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:43.465409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4fe63-eea1-41da-83e4-5dcd71b1c657 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:43.484864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 39/84] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:34:43 2022] PUT /identity/v3/projects/72d02777c6694301959a12c02452adf3/users/9e8fef123072495eabcb616db3e670a6/roles/af3529d63bae4650be7b8dc67b3ed527 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:34:43.513588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:43.561417 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb0ffe-b9d4-48a1-86ee-ff522dbf6b5e None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:43.613622 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee8486-8439-4719-95bd-4ca16f3977fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:43.614698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee8486-8439-4719-95bd-4ca16f3977fb None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:43.615192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee8486-8439-4719-95bd-4ca16f3977fb None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:43.616359 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee8486-8439-4719-95bd-4ca16f3977fb None admin] PATH_INFO: `/v3/projects/266f772f871a4f678b085a45bf073f3c` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:43.619048 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee8486-8439-4719-95bd-4ca16f3977fb None admin] RBAC: Authorizing `identity:get_project(project_id=266f772f871a4f678b085a45bf073f3c)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:43.621034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee8486-8439-4719-95bd-4ca16f3977fb None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:43.623095 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 46/85] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:34:43 2022] GET /identity/v3/projects/266f772f871a4f678b085a45bf073f3c => generated 312 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:43.630766 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4fe63-eea1-41da-83e4-5dcd71b1c657 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:43.780129 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4fe63-eea1-41da-83e4-5dcd71b1c657 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:43.816738 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:43.826389 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1063b26-e588-4a66-b632-1974bacbb8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:43.827248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1063b26-e588-4a66-b632-1974bacbb8a8 None admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:43.827640 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1063b26-e588-4a66-b632-1974bacbb8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:43.828026 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1063b26-e588-4a66-b632-1974bacbb8a8 None admin] PATH_INFO: `/v3/projects/266f772f871a4f678b085a45bf073f3c/users/9e8fef123072495eabcb616db3e670a6/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:43.832385 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1063b26-e588-4a66-b632-1974bacbb8a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=266f772f871a4f678b085a45bf073f3c, user_id=9e8fef123072495eabcb616db3e670a6, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:43.834021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1063b26-e588-4a66-b632-1974bacbb8a8 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:43.857221 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 40/86] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:34:43 2022] PUT /identity/v3/projects/266f772f871a4f678b085a45bf073f3c/users/9e8fef123072495eabcb616db3e670a6/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:34:45.397798 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83421ff-a71b-4db9-b4cc-1c8e792ced35 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:45.398347 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83421ff-a71b-4db9-b4cc-1c8e792ced35 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:45.398629 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83421ff-a71b-4db9-b4cc-1c8e792ced35 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:45.400021 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 47/87] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:45.406004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a100656-708a-4492-91b7-4c41833f0203 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:45.406480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a100656-708a-4492-91b7-4c41833f0203 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:45.406550 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a100656-708a-4492-91b7-4c41833f0203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:45.437200 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-0a100656-708a-4492-91b7-4c41833f0203 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:45.438449 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-0a100656-708a-4492-91b7-4c41833f0203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:45.541877 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 41/88] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:46.111504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83421ff-a71b-4db9-b4cc-1c8e792ced35 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:46.236541 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83421ff-a71b-4db9-b4cc-1c8e792ced35 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:46.280639 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c526b1e-e107-4f54-86c2-d258f9923e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:46.281652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c526b1e-e107-4f54-86c2-d258f9923e81 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:46.282144 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c526b1e-e107-4f54-86c2-d258f9923e81 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:46.282620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c526b1e-e107-4f54-86c2-d258f9923e81 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:46.286259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:46.296985 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c526b1e-e107-4f54-86c2-d258f9923e81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:34:46.302325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-7c526b1e-e107-4f54-86c2-d258f9923e81 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:34:46.302325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:46.302325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:46.302325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:46.302325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:46.302325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:46.302325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:46.302325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:46.302325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:46.302325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:46.302325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:46.302325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:46.302325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:34:46.302325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:34:46.302325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:34:46.302325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:34:46.303477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:34:46.303477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:34:46.303477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:34:46.303477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:34:46.303477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:46.303477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:46.303477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:46.303477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:46.303477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:46.303477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:46.303477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:46.303477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:46.303477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:46.303477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:46.303477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:46.303477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:46.304281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:46.304281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:46.304281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:46.304281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:46.304281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:34:46.304281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:34:46.304281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:34:46.304281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:34:46.304281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:34:46.304281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:34:46.304281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:34:46.304281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:34:46.306648 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 48/89] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:34:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:34:46.312608 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a100656-708a-4492-91b7-4c41833f0203 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:46.378774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104082c1-76b1-47b5-a6e2-1f036309d363 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:46.379408 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104082c1-76b1-47b5-a6e2-1f036309d363 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:46.379889 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104082c1-76b1-47b5-a6e2-1f036309d363 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:46.383658 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 49/90] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:46.390783 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4fa8d-b92d-4127-98f4-c7db181056b0 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:46.391395 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4fa8d-b92d-4127-98f4-c7db181056b0 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:46.391874 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4fa8d-b92d-4127-98f4-c7db181056b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:46.421841 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-64e4fa8d-b92d-4127-98f4-c7db181056b0 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:46.427181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-64e4fa8d-b92d-4127-98f4-c7db181056b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:46.520446 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a100656-708a-4492-91b7-4c41833f0203 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:46.539571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 50/91] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:46.564490 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:46.597388 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638f56a-faae-4480-a35a-316de6d2c9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:46.598023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638f56a-faae-4480-a35a-316de6d2c9a1 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:46.598304 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638f56a-faae-4480-a35a-316de6d2c9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:46.598581 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638f56a-faae-4480-a35a-316de6d2c9a1 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:46.599265 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1638f56a-faae-4480-a35a-316de6d2c9a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:46.599568 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1638f56a-faae-4480-a35a-316de6d2c9a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:46.601324 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1638f56a-faae-4480-a35a-316de6d2c9a1 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:46.614339 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 42/92] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:34:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:34:46.621213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4fa8d-b92d-4127-98f4-c7db181056b0 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:46.653351 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e9881-d398-4d59-ab43-d2c2b5f4705a None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:46.653669 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e9881-d398-4d59-ab43-d2c2b5f4705a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:46.653902 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e9881-d398-4d59-ab43-d2c2b5f4705a None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:46.655358 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 43/93] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:34:46.678666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d721170-aacd-468f-b116-d15e63eb364f None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:46.678971 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d721170-aacd-468f-b116-d15e63eb364f None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:46.679178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d721170-aacd-468f-b116-d15e63eb364f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:46.717258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-0d721170-aacd-468f-b116-d15e63eb364f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:46.718882 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-0d721170-aacd-468f-b116-d15e63eb364f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:46.747179 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4fa8d-b92d-4127-98f4-c7db181056b0 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:46.777427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:46.788970 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d796ed9-28aa-4bdf-a34a-ecad8ce11538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:46.789991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d796ed9-28aa-4bdf-a34a-ecad8ce11538 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:46.790478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d796ed9-28aa-4bdf-a34a-ecad8ce11538 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:46.790956 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d796ed9-28aa-4bdf-a34a-ecad8ce11538 None admin] PATH_INFO: `/v3/users/9e8fef123072495eabcb616db3e670a6` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:46.792415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d796ed9-28aa-4bdf-a34a-ecad8ce11538 None admin] RBAC: Authorizing `identity:get_user(user_id=9e8fef123072495eabcb616db3e670a6)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:46.794164 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d796ed9-28aa-4bdf-a34a-ecad8ce11538 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:46.796264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 51/94] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:34:46 2022] GET /identity/v3/users/9e8fef123072495eabcb616db3e670a6 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:46.804103 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d796ed9-28aa-4bdf-a34a-ecad8ce11538 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:46.851090 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 44/95] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:34:46.935685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d796ed9-28aa-4bdf-a34a-ecad8ce11538 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:46.966149 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d721170-aacd-468f-b116-d15e63eb364f None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:46.974942 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d019ec9-3194-470a-84c1-80aa14e6e87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:46.976699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d019ec9-3194-470a-84c1-80aa14e6e87f None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:46.977197 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d019ec9-3194-470a-84c1-80aa14e6e87f None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:46.977669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d019ec9-3194-470a-84c1-80aa14e6e87f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:47.050968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:47.061828 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d019ec9-3194-470a-84c1-80aa14e6e87f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:47.063549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d019ec9-3194-470a-84c1-80aa14e6e87f None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:47.102794 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d721170-aacd-468f-b116-d15e63eb364f None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:47.123335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:47.157218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc3689-29c2-4616-87fe-956ff8dc3941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:47.158256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc3689-29c2-4616-87fe-956ff8dc3941 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:47.158789 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc3689-29c2-4616-87fe-956ff8dc3941 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:47.159167 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc3689-29c2-4616-87fe-956ff8dc3941 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:47.173547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc3689-29c2-4616-87fe-956ff8dc3941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:34:47.183593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-4afc3689-29c2-4616-87fe-956ff8dc3941 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:34:47.183593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:47.183593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:47.183593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:47.183593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:47.183593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:47.183593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:47.183593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:47.183593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:47.183593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:47.183593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:47.183593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:47.183593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:34:47.183593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:34:47.183593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:34:47.183593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:34:47.184945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:34:47.184945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:34:47.184945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:34:47.184945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:34:47.184945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:47.184945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:47.184945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:47.184945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:47.184945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:47.184945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:47.184945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:47.184945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:47.184945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:47.184945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:47.184945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:47.184945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:47.186279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:47.186279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:47.186279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:47.186279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:47.186279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:34:47.186279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:34:47.186279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:34:47.186279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:34:47.186279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:34:47.186279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:34:47.186279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:34:47.186279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:47.187757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 45/96] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:34:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:34:47.197568 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc3689-29c2-4616-87fe-956ff8dc3941 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:47.291777 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 52/97] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:34:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:34:47.297301 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d019ec9-3194-470a-84c1-80aa14e6e87f None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:47.401882 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc3689-29c2-4616-87fe-956ff8dc3941 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:47.450364 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8705e37-2825-4860-a569-9c26f1555ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:47.451000 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8705e37-2825-4860-a569-9c26f1555ff1 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:47.451286 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8705e37-2825-4860-a569-9c26f1555ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:47.451567 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8705e37-2825-4860-a569-9c26f1555ff1 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:47.452294 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8705e37-2825-4860-a569-9c26f1555ff1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:47.452607 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8705e37-2825-4860-a569-9c26f1555ff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:47.454281 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8705e37-2825-4860-a569-9c26f1555ff1 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:47.465614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 46/98] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:34:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:34:47.466695 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8705e37-2825-4860-a569-9c26f1555ff1 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:47.482027 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:47.492231 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d019ec9-3194-470a-84c1-80aa14e6e87f None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:47.521024 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:47.530337 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00715194-4609-4a8c-a862-498c06bb13c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:47.530998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00715194-4609-4a8c-a862-498c06bb13c0 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:47.531281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00715194-4609-4a8c-a862-498c06bb13c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:47.531556 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00715194-4609-4a8c-a862-498c06bb13c0 None admin] PATH_INFO: `/v3/projects/72d02777c6694301959a12c02452adf3` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:47.532836 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00715194-4609-4a8c-a862-498c06bb13c0 None admin] RBAC: Authorizing `identity:get_project(project_id=72d02777c6694301959a12c02452adf3)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:47.534324 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00715194-4609-4a8c-a862-498c06bb13c0 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:47.536319 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 53/99] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:34:47 2022] GET /identity/v3/projects/72d02777c6694301959a12c02452adf3 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:47.538127 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e052add-7bff-4765-beee-f7c24cf3e2c9 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:47.538595 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e052add-7bff-4765-beee-f7c24cf3e2c9 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:47.538962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e052add-7bff-4765-beee-f7c24cf3e2c9 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:47.540263 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 54/100] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:47.540983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e052add-7bff-4765-beee-f7c24cf3e2c9 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:47.612751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8705e37-2825-4860-a569-9c26f1555ff1 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:47.656493 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248b3b0-b7a2-4fee-90ae-02f8600d7325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:47.657049 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248b3b0-b7a2-4fee-90ae-02f8600d7325 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:47.657329 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248b3b0-b7a2-4fee-90ae-02f8600d7325 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:47.657603 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248b3b0-b7a2-4fee-90ae-02f8600d7325 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:47.664890 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e052add-7bff-4765-beee-f7c24cf3e2c9 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:47.671853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248b3b0-b7a2-4fee-90ae-02f8600d7325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:34:47.677376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-3248b3b0-b7a2-4fee-90ae-02f8600d7325 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:34:47.677376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:47.677376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:47.677376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:47.677376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:47.677376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:47.677376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:47.677376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:47.677376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:47.677376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:47.677376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:47.677376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:47.677376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:34:47.677376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:34:47.677376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:34:47.677376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:34:47.678345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:34:47.678345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:34:47.678345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:34:47.678345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:34:47.678345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:47.678345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:47.678345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:47.678345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:47.678345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:47.678345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:47.678345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:47.678345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:47.678345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:47.678345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:47.678345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:47.678345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:47.679122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:47.679122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:47.679122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:47.679122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:47.679122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:34:47.679122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:34:47.679122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:34:47.679122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:34:47.679122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:34:47.679122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:34:47.679122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:34:47.679122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:47.679706 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 47/101] 10.222.0.36 () {56 vars in 1114 bytes} [Sun Oct 30 00:34:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:34:47.680291 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248b3b0-b7a2-4fee-90ae-02f8600d7325 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:47.707314 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266716d9-c7ba-48e9-a675-6e58e845c381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:47.707958 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266716d9-c7ba-48e9-a675-6e58e845c381 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:47.708276 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266716d9-c7ba-48e9-a675-6e58e845c381 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:47.708555 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266716d9-c7ba-48e9-a675-6e58e845c381 None admin] PATH_INFO: `/v3/users/245a44aa1d7f43fcad2e727557036aab` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:47.709785 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266716d9-c7ba-48e9-a675-6e58e845c381 None admin] RBAC: Authorizing `identity:get_user(user_id=245a44aa1d7f43fcad2e727557036aab)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:47.711767 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266716d9-c7ba-48e9-a675-6e58e845c381 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:47.713439 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 55/102] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:34:47 2022] GET /identity/v3/users/245a44aa1d7f43fcad2e727557036aab => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:47.714594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf904f2f-2fed-426c-9f2d-3cdbfaa2b9e4 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:47.714879 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf904f2f-2fed-426c-9f2d-3cdbfaa2b9e4 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:47.715154 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf904f2f-2fed-426c-9f2d-3cdbfaa2b9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:47.733382 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-bf904f2f-2fed-426c-9f2d-3cdbfaa2b9e4 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:47.734603 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-bf904f2f-2fed-426c-9f2d-3cdbfaa2b9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:47.759536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:47.791542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:47.800535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248b3b0-b7a2-4fee-90ae-02f8600d7325 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:47.807089 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 56/103] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:47.808485 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b6e90-9a2d-4f48-b1da-7b19e2ddeecc None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:47.808781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b6e90-9a2d-4f48-b1da-7b19e2ddeecc None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:47.809065 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b6e90-9a2d-4f48-b1da-7b19e2ddeecc None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:47.810371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 57/104] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:47.811443 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca86fc-789b-45b8-911f-be0804a78e7b None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:47.811725 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca86fc-789b-45b8-911f-be0804a78e7b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:47.812073 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca86fc-789b-45b8-911f-be0804a78e7b None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:47.813227 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 58/105] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:47.813929 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca86fc-789b-45b8-911f-be0804a78e7b None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:47.845533 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993cb79-4af1-4386-bfa0-3dcc5b7d8811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:47.846151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993cb79-4af1-4386-bfa0-3dcc5b7d8811 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:47.846429 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993cb79-4af1-4386-bfa0-3dcc5b7d8811 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:47.846704 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993cb79-4af1-4386-bfa0-3dcc5b7d8811 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:47.847443 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993cb79-4af1-4386-bfa0-3dcc5b7d8811 None admin] RBAC: Adding query filter params (role.id=2e0593ed85a14197a0eaf37135a01787, scope.project.id=72d02777c6694301959a12c02452adf3, user.id=9e8fef123072495eabcb616db3e670a6) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:47.847756 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993cb79-4af1-4386-bfa0-3dcc5b7d8811 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:47.849262 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993cb79-4af1-4386-bfa0-3dcc5b7d8811 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:47.864140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 48/106] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:34:47 2022] GET /identity/v3/role_assignments?user.id=9e8fef123072495eabcb616db3e670a6&scope.project.id=72d02777c6694301959a12c02452adf3&role.id=2e0593ed85a14197a0eaf37135a01787 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:47.864140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993cb79-4af1-4386-bfa0-3dcc5b7d8811 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:47.979129 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca86fc-789b-45b8-911f-be0804a78e7b None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:48.005710 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993cb79-4af1-4386-bfa0-3dcc5b7d8811 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:48.034996 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a770139-1b10-456a-9f9f-e4d69bfc3686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:48.035702 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a770139-1b10-456a-9f9f-e4d69bfc3686 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:48.035995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a770139-1b10-456a-9f9f-e4d69bfc3686 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:48.036422 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a770139-1b10-456a-9f9f-e4d69bfc3686 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:48.037124 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a770139-1b10-456a-9f9f-e4d69bfc3686 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:48.037438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a770139-1b10-456a-9f9f-e4d69bfc3686 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:48.039696 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a770139-1b10-456a-9f9f-e4d69bfc3686 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:48.051262 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 59/107] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:34:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:48.053877 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28545c05-5587-4717-a40c-e6e8286c5d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:48.054242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48d9a4-81dd-4bff-8338-69034d09652a None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:48.054770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48d9a4-81dd-4bff-8338-69034d09652a None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:48.055238 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48d9a4-81dd-4bff-8338-69034d09652a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:48.055487 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28545c05-5587-4717-a40c-e6e8286c5d04 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:48.055951 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28545c05-5587-4717-a40c-e6e8286c5d04 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:48.056457 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28545c05-5587-4717-a40c-e6e8286c5d04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:48.078968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-df48d9a4-81dd-4bff-8338-69034d09652a None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:48.080222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-df48d9a4-81dd-4bff-8338-69034d09652a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:48.096454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:48.100232 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:48.147486 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28545c05-5587-4717-a40c-e6e8286c5d04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:48.149214 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28545c05-5587-4717-a40c-e6e8286c5d04 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:48.161032 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 60/108] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:48.162317 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b4127-ee84-42e7-a0bc-fad163666da0 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:48.162937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b4127-ee84-42e7-a0bc-fad163666da0 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:48.162937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b4127-ee84-42e7-a0bc-fad163666da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:48.181886 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-095b4127-ee84-42e7-a0bc-fad163666da0 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:48.183329 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-095b4127-ee84-42e7-a0bc-fad163666da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:48.268328 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 61/109] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:48.268949 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b4127-ee84-42e7-a0bc-fad163666da0 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:48.290901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 49/110] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:34:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:34:48.296293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28545c05-5587-4717-a40c-e6e8286c5d04 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:48.372479 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:48.391822 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:48.415295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b4127-ee84-42e7-a0bc-fad163666da0 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:48.438131 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28545c05-5587-4717-a40c-e6e8286c5d04 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:48.455143 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ae64c-8190-4d39-aa44-acddd7e36824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:48.455704 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ae64c-8190-4d39-aa44-acddd7e36824 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:48.455983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ae64c-8190-4d39-aa44-acddd7e36824 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:48.456303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ae64c-8190-4d39-aa44-acddd7e36824 None admin] PATH_INFO: `/v3/users/9e8fef123072495eabcb616db3e670a6` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:48.457477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ae64c-8190-4d39-aa44-acddd7e36824 None admin] RBAC: Authorizing `identity:get_user(user_id=9e8fef123072495eabcb616db3e670a6)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:48.459086 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ae64c-8190-4d39-aa44-acddd7e36824 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:48.461634 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 62/111] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:34:48 2022] GET /identity/v3/users/9e8fef123072495eabcb616db3e670a6 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:48.462775 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ae64c-8190-4d39-aa44-acddd7e36824 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:48.486739 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a96ca-05f8-4dca-8b22-de0075be6d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:48.487452 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a96ca-05f8-4dca-8b22-de0075be6d9b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:48.487765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a96ca-05f8-4dca-8b22-de0075be6d9b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:48.488046 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a96ca-05f8-4dca-8b22-de0075be6d9b None admin] PATH_INFO: `/v3/domains/default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:48.490156 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a96ca-05f8-4dca-8b22-de0075be6d9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:48.490833 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a96ca-05f8-4dca-8b22-de0075be6d9b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:48.492717 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 50/112] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:34:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:48.497883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a96ca-05f8-4dca-8b22-de0075be6d9b None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:48.575383 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ae64c-8190-4d39-aa44-acddd7e36824 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:48.607175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a96ca-05f8-4dca-8b22-de0075be6d9b None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:48.613325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec36087-a199-4013-8658-8ba3bcf82b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:48.613975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec36087-a199-4013-8658-8ba3bcf82b37 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:48.614259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec36087-a199-4013-8658-8ba3bcf82b37 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:48.614536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec36087-a199-4013-8658-8ba3bcf82b37 None admin] PATH_INFO: `/v3/projects/72d02777c6694301959a12c02452adf3` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:48.616282 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec36087-a199-4013-8658-8ba3bcf82b37 None admin] RBAC: Authorizing `identity:get_project(project_id=72d02777c6694301959a12c02452adf3)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:48.618038 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec36087-a199-4013-8658-8ba3bcf82b37 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:48.619942 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 63/113] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:34:48 2022] GET /identity/v3/projects/72d02777c6694301959a12c02452adf3 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:48.620602 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec36087-a199-4013-8658-8ba3bcf82b37 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:48.639969 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:48.642597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:48.651577 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e948e-7375-4a97-8651-f9cf3423caa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:48.652234 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e948e-7375-4a97-8651-f9cf3423caa4 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:48.652518 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e948e-7375-4a97-8651-f9cf3423caa4 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:48.652796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e948e-7375-4a97-8651-f9cf3423caa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:48.731723 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e948e-7375-4a97-8651-f9cf3423caa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:48.732154 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec36087-a199-4013-8658-8ba3bcf82b37 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:48.732764 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e948e-7375-4a97-8651-f9cf3423caa4 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:48.767284 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134a3f7-3430-4f23-bda5-7544d5280ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:48.767916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134a3f7-3430-4f23-bda5-7544d5280ebd None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:48.768225 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134a3f7-3430-4f23-bda5-7544d5280ebd None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:48.768504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134a3f7-3430-4f23-bda5-7544d5280ebd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:48.783167 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a134a3f7-3430-4f23-bda5-7544d5280ebd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:34:48.789480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-a134a3f7-3430-4f23-bda5-7544d5280ebd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:34:48.789480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:48.789480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:48.789480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:48.789480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:48.789480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:48.789480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:48.789480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:48.789480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:48.789480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:48.789480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:48.789480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:48.789480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:34:48.789480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:34:48.789480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:34:48.789480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:34:48.790427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:34:48.790427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:34:48.790427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:34:48.790427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:34:48.790427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:48.790427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:48.790427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:48.790427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:48.790427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:48.790427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:48.790427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:48.790427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:48.790427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:48.790427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:48.790427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:48.790427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:48.791222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:48.791222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:48.791222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:48.791222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:48.791222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:34:48.791222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:34:48.791222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:34:48.791222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:34:48.791222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:34:48.791222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:34:48.791222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:34:48.791222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:34:48.791222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 64/114] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:34:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:34:48.791883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134a3f7-3430-4f23-bda5-7544d5280ebd None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:48.855042 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 51/115] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:34:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:34:48.855955 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e948e-7375-4a97-8651-f9cf3423caa4 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:48.877478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:48.891180 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:48.907284 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134a3f7-3430-4f23-bda5-7544d5280ebd None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:48.949783 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5de726-dcb9-41f9-9399-a84ecea4106d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:48.950405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5de726-dcb9-41f9-9399-a84ecea4106d None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:48.950691 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5de726-dcb9-41f9-9399-a84ecea4106d None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:48.950971 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5de726-dcb9-41f9-9399-a84ecea4106d None admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:48.951696 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5de726-dcb9-41f9-9399-a84ecea4106d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:48.953284 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5de726-dcb9-41f9-9399-a84ecea4106d None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:48.983927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e948e-7375-4a97-8651-f9cf3423caa4 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:48.984748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 65/116] 10.222.0.36 () {60 vars in 1141 bytes} [Sun Oct 30 00:34:48 2022] POST /identity/v3/projects => generated 302 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:34:48.985601 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5de726-dcb9-41f9-9399-a84ecea4106d None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:49.032533 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e4950-41a2-46fb-9aa6-97552333b550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:49.033199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e4950-41a2-46fb-9aa6-97552333b550 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:49.033485 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e4950-41a2-46fb-9aa6-97552333b550 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:49.033761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e4950-41a2-46fb-9aa6-97552333b550 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:49.047633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e4950-41a2-46fb-9aa6-97552333b550 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:34:49.052939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-f17e4950-41a2-46fb-9aa6-97552333b550 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:34:49.052939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:49.052939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:49.052939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:49.052939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:49.052939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:49.052939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:49.052939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:49.052939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:49.052939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:49.052939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:49.052939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:49.052939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:34:49.052939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:34:49.052939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:34:49.052939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:34:49.054537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:34:49.054537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:34:49.054537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:34:49.054537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:34:49.054537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:49.054537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:49.054537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:49.054537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:49.054537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:49.054537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:49.054537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:49.054537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:49.054537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:49.054537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:49.054537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:49.054537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:49.055425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:49.055425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:49.055425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:49.055425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:49.055425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:34:49.055425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:34:49.055425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:34:49.055425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:34:49.055425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:34:49.055425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:34:49.055425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:34:49.055425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:49.055425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 52/117] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:34:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:34:49.060362 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e4950-41a2-46fb-9aa6-97552333b550 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:49.122970 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5de726-dcb9-41f9-9399-a84ecea4106d None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:49.163911 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:49.171645 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1ab7a-899f-4968-95e9-3a891df4bd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:49.172256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:49.173631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1ab7a-899f-4968-95e9-3a891df4bd54 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:49.174189 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1ab7a-899f-4968-95e9-3a891df4bd54 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:49.174653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1ab7a-899f-4968-95e9-3a891df4bd54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:49.175653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1ab7a-899f-4968-95e9-3a891df4bd54 None admin] RBAC: Adding query filter params (role.id=980241d45fdb435d9f32c434057c71e4, scope.project.id=72d02777c6694301959a12c02452adf3, user.id=245a44aa1d7f43fcad2e727557036aab) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:49.176166 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1ab7a-899f-4968-95e9-3a891df4bd54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:49.177661 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1ab7a-899f-4968-95e9-3a891df4bd54 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:49.186022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 66/118] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:34:48 2022] GET /identity/v3/role_assignments?user.id=245a44aa1d7f43fcad2e727557036aab&scope.project.id=72d02777c6694301959a12c02452adf3&role.id=980241d45fdb435d9f32c434057c71e4 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:49.187427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1ab7a-899f-4968-95e9-3a891df4bd54 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:49.191287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e4950-41a2-46fb-9aa6-97552333b550 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:49.255531 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efc5f1-ceea-4c3e-b9f5-fd782155ae4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:49.257530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efc5f1-ceea-4c3e-b9f5-fd782155ae4b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:49.257821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efc5f1-ceea-4c3e-b9f5-fd782155ae4b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:49.258102 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efc5f1-ceea-4c3e-b9f5-fd782155ae4b None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:49.258817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efc5f1-ceea-4c3e-b9f5-fd782155ae4b None admin] RBAC: Adding query filter params (name=reader) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:49.259123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efc5f1-ceea-4c3e-b9f5-fd782155ae4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:49.260607 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efc5f1-ceea-4c3e-b9f5-fd782155ae4b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:49.279136 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 53/119] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:34:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:49.279438 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efc5f1-ceea-4c3e-b9f5-fd782155ae4b None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:49.353369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1ab7a-899f-4968-95e9-3a891df4bd54 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:49.405894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166178c-cd72-4f7f-8e05-c3dd45bc25fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:49.406705 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166178c-cd72-4f7f-8e05-c3dd45bc25fb None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:49.406991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166178c-cd72-4f7f-8e05-c3dd45bc25fb None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:49.407536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166178c-cd72-4f7f-8e05-c3dd45bc25fb None admin] PATH_INFO: `/v3/projects/72d02777c6694301959a12c02452adf3` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:49.411092 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5166178c-cd72-4f7f-8e05-c3dd45bc25fb None admin] RBAC: Authorizing `identity:get_project(project_id=72d02777c6694301959a12c02452adf3)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:49.411359 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5166178c-cd72-4f7f-8e05-c3dd45bc25fb None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:49.412168 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efc5f1-ceea-4c3e-b9f5-fd782155ae4b None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:49.414647 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 67/120] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:34:49 2022] GET /identity/v3/projects/72d02777c6694301959a12c02452adf3 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:49.415587 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166178c-cd72-4f7f-8e05-c3dd45bc25fb None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:49.456270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d2c45-bca9-44d3-bdd0-94505e350dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:49.456901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d2c45-bca9-44d3-bdd0-94505e350dce None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:49.457185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d2c45-bca9-44d3-bdd0-94505e350dce None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:49.457462 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d2c45-bca9-44d3-bdd0-94505e350dce None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:49.458150 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d2c45-bca9-44d3-bdd0-94505e350dce None admin] RBAC: Adding query filter params (name=member) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:49.458458 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d2c45-bca9-44d3-bdd0-94505e350dce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:49.463049 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d2c45-bca9-44d3-bdd0-94505e350dce None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:49.477037 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 54/121] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:34:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:49.477981 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d2c45-bca9-44d3-bdd0-94505e350dce None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:49.496958 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:49.498169 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:49.541938 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166178c-cd72-4f7f-8e05-c3dd45bc25fb None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:49.577678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acce770-30a9-4fc5-b840-e613eada14ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:49.578303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acce770-30a9-4fc5-b840-e613eada14ac None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:49.578595 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acce770-30a9-4fc5-b840-e613eada14ac None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:49.578880 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acce770-30a9-4fc5-b840-e613eada14ac None admin] PATH_INFO: `/v3/users/fded43439072492ea7816449fe30d3bf` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:49.595669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acce770-30a9-4fc5-b840-e613eada14ac None admin] RBAC: Authorizing `identity:get_user(user_id=fded43439072492ea7816449fe30d3bf)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:49.597234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acce770-30a9-4fc5-b840-e613eada14ac None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:49.606276 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 68/122] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:34:49 2022] GET /identity/v3/users/fded43439072492ea7816449fe30d3bf => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:49.606276 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acce770-30a9-4fc5-b840-e613eada14ac None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:49.613218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d2c45-bca9-44d3-bdd0-94505e350dce None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:49.654000 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b47ee-7a38-48d9-9a03-4b33331d6a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:49.654621 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b47ee-7a38-48d9-9a03-4b33331d6a2d None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:49.654906 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b47ee-7a38-48d9-9a03-4b33331d6a2d None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:49.655181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b47ee-7a38-48d9-9a03-4b33331d6a2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:49.655982 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b47ee-7a38-48d9-9a03-4b33331d6a2d None admin] RBAC: Adding query filter params (role.id=af3529d63bae4650be7b8dc67b3ed527, scope.project.id=72d02777c6694301959a12c02452adf3, user.id=9e8fef123072495eabcb616db3e670a6) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:49.656273 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b47ee-7a38-48d9-9a03-4b33331d6a2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:49.657736 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b47ee-7a38-48d9-9a03-4b33331d6a2d None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:49.665213 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 55/123] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:34:49 2022] GET /identity/v3/role_assignments?user.id=9e8fef123072495eabcb616db3e670a6&scope.project.id=72d02777c6694301959a12c02452adf3&role.id=af3529d63bae4650be7b8dc67b3ed527 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:49.677179 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b47ee-7a38-48d9-9a03-4b33331d6a2d None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:49.734036 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acce770-30a9-4fc5-b840-e613eada14ac None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:49.760926 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:49.787220 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e7622-7fe7-4397-895d-4b530495547b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:49.788976 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e7622-7fe7-4397-895d-4b530495547b None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:49.789765 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e7622-7fe7-4397-895d-4b530495547b None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:49.790335 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e7622-7fe7-4397-895d-4b530495547b None admin] PATH_INFO: `/v3/users/9e8fef123072495eabcb616db3e670a6` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:49.792927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e7622-7fe7-4397-895d-4b530495547b None admin] RBAC: Authorizing `identity:get_user(user_id=9e8fef123072495eabcb616db3e670a6)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:49.796027 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e7622-7fe7-4397-895d-4b530495547b None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:49.805525 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:49.806159 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 69/124] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:34:49 2022] GET /identity/v3/users/9e8fef123072495eabcb616db3e670a6 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:49.807799 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e7622-7fe7-4397-895d-4b530495547b None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:49.837993 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b47ee-7a38-48d9-9a03-4b33331d6a2d None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:49.888238 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c88e04-1c39-493c-8011-581469ceb200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:49.888238 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c88e04-1c39-493c-8011-581469ceb200 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:49.888238 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c88e04-1c39-493c-8011-581469ceb200 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:49.888238 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c88e04-1c39-493c-8011-581469ceb200 None admin] PATH_INFO: `/v3/projects/72d02777c6694301959a12c02452adf3` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:49.888624 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c88e04-1c39-493c-8011-581469ceb200 None admin] RBAC: Authorizing `identity:get_project(project_id=72d02777c6694301959a12c02452adf3)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:49.888624 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c88e04-1c39-493c-8011-581469ceb200 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:49.891892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 56/125] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:34:49 2022] GET /identity/v3/projects/72d02777c6694301959a12c02452adf3 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:49.896048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c88e04-1c39-493c-8011-581469ceb200 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:49.963125 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e7622-7fe7-4397-895d-4b530495547b None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:50.003868 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a5a15-f989-487e-8a20-3173eca6a4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:50.004667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a5a15-f989-487e-8a20-3173eca6a4ef None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:50.004953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a5a15-f989-487e-8a20-3173eca6a4ef None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:50.005232 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a5a15-f989-487e-8a20-3173eca6a4ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:50.032772 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c88e04-1c39-493c-8011-581469ceb200 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:50.069742 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817de5e-6d43-4832-9776-69db43c8a46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:50.070606 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817de5e-6d43-4832-9776-69db43c8a46f None admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:50.070890 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817de5e-6d43-4832-9776-69db43c8a46f None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:50.071163 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817de5e-6d43-4832-9776-69db43c8a46f None admin] PATH_INFO: `/v3/projects/72d02777c6694301959a12c02452adf3/users/fded43439072492ea7816449fe30d3bf/roles/2e58f1f4fb154236982461a05ee16601` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:50.073226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817de5e-6d43-4832-9776-69db43c8a46f None admin] RBAC: Authorizing `identity:create_grant(project_id=72d02777c6694301959a12c02452adf3, user_id=fded43439072492ea7816449fe30d3bf, role_id=2e58f1f4fb154236982461a05ee16601)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:50.074692 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817de5e-6d43-4832-9776-69db43c8a46f None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:50.075024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:50.090175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 57/126] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:34:49 2022] PUT /identity/v3/projects/72d02777c6694301959a12c02452adf3/users/fded43439072492ea7816449fe30d3bf/roles/2e58f1f4fb154236982461a05ee16601 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:34:50.093237 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a5a15-f989-487e-8a20-3173eca6a4ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:50.095377 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a5a15-f989-487e-8a20-3173eca6a4ef None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:50.221025 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 70/127] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:34:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 30 00:34:50.222274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817de5e-6d43-4832-9776-69db43c8a46f None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:50.225614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:50.343708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817de5e-6d43-4832-9776-69db43c8a46f None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:50.381139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b64480-78ec-469c-8824-e0cd186f8e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:50.382186 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b64480-78ec-469c-8824-e0cd186f8e84 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:50.382338 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b64480-78ec-469c-8824-e0cd186f8e84 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:50.382618 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b64480-78ec-469c-8824-e0cd186f8e84 None admin] PATH_INFO: `/v3/projects/266f772f871a4f678b085a45bf073f3c` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:50.383796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b64480-78ec-469c-8824-e0cd186f8e84 None admin] RBAC: Authorizing `identity:get_project(project_id=266f772f871a4f678b085a45bf073f3c)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:50.385311 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b64480-78ec-469c-8824-e0cd186f8e84 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:50.387168 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 58/128] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:34:50 2022] GET /identity/v3/projects/266f772f871a4f678b085a45bf073f3c => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:50.393302 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a5a15-f989-487e-8a20-3173eca6a4ef None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:50.493378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:50.516148 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a5a15-f989-487e-8a20-3173eca6a4ef None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:50.555725 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c63176-19c9-43a0-9555-1a78bf3c1208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:50.556378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c63176-19c9-43a0-9555-1a78bf3c1208 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:50.556663 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c63176-19c9-43a0-9555-1a78bf3c1208 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:50.556938 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c63176-19c9-43a0-9555-1a78bf3c1208 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:50.557707 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c63176-19c9-43a0-9555-1a78bf3c1208 None admin] RBAC: Adding query filter params (role.id=2e0593ed85a14197a0eaf37135a01787, scope.project.id=266f772f871a4f678b085a45bf073f3c, user.id=9e8fef123072495eabcb616db3e670a6) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:50.558021 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c63176-19c9-43a0-9555-1a78bf3c1208 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:50.559496 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c63176-19c9-43a0-9555-1a78bf3c1208 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:50.565950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 71/129] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:34:50 2022] GET /identity/v3/role_assignments?user.id=9e8fef123072495eabcb616db3e670a6&scope.project.id=266f772f871a4f678b085a45bf073f3c&role.id=2e0593ed85a14197a0eaf37135a01787 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:34:51.316829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cca57-77b4-402f-8f59-d1d496135d0f None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:51.317346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cca57-77b4-402f-8f59-d1d496135d0f None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:51.317451 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cca57-77b4-402f-8f59-d1d496135d0f None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:51.318714 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 59/130] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:51.324848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e7299-c563-4db5-9286-210c23fd8675 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:51.325303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e7299-c563-4db5-9286-210c23fd8675 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:51.325387 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e7299-c563-4db5-9286-210c23fd8675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:51.344505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-9b7e7299-c563-4db5-9286-210c23fd8675 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:51.345737 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-9b7e7299-c563-4db5-9286-210c23fd8675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:51.415024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 72/131] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:51.687329 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cca57-77b4-402f-8f59-d1d496135d0f None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:51.764702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:51.798069 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cca57-77b4-402f-8f59-d1d496135d0f None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:51.833914 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142d287-b500-49cc-a5f8-58d16a993756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:51.834557 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142d287-b500-49cc-a5f8-58d16a993756 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:51.834872 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142d287-b500-49cc-a5f8-58d16a993756 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:51.835586 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142d287-b500-49cc-a5f8-58d16a993756 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:51.836310 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142d287-b500-49cc-a5f8-58d16a993756 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:51.837764 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142d287-b500-49cc-a5f8-58d16a993756 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:51.839358 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 60/132] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:34:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:51.845204 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e7299-c563-4db5-9286-210c23fd8675 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:51.936025 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:51.956259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e7299-c563-4db5-9286-210c23fd8675 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:51.991637 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f34359-9271-4f4d-9037-c964c821df62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:51.992281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f34359-9271-4f4d-9037-c964c821df62 None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:51.992568 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f34359-9271-4f4d-9037-c964c821df62 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:51.992838 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f34359-9271-4f4d-9037-c964c821df62 None admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:51.993493 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f34359-9271-4f4d-9037-c964c821df62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:51.994969 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f34359-9271-4f4d-9037-c964c821df62 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:52.029695 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 73/133] 10.222.0.36 () {60 vars in 1132 bytes} [Sun Oct 30 00:34:51 2022] POST /identity/v3/users => generated 283 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:34:52.312624 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfb0bb-46c7-4b64-8c30-824f4174521b None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:52.312979 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfb0bb-46c7-4b64-8c30-824f4174521b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:52.313427 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfb0bb-46c7-4b64-8c30-824f4174521b None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:52.315200 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 61/134] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:52.321874 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c97e8b-7e30-4861-a910-4536d935d97e None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:52.322165 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c97e8b-7e30-4861-a910-4536d935d97e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:52.322520 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c97e8b-7e30-4861-a910-4536d935d97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:52.342609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-80c97e8b-7e30-4861-a910-4536d935d97e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:52.343834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-80c97e8b-7e30-4861-a910-4536d935d97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:52.428926 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 74/135] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:52.705712 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfb0bb-46c7-4b64-8c30-824f4174521b None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:52.826464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfb0bb-46c7-4b64-8c30-824f4174521b None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:52.878199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eafae6-b515-4f43-b776-457bd9a323de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:52.879122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eafae6-b515-4f43-b776-457bd9a323de None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:52.879526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eafae6-b515-4f43-b776-457bd9a323de None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:52.879925 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eafae6-b515-4f43-b776-457bd9a323de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:52.896138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eafae6-b515-4f43-b776-457bd9a323de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:34:52.899394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-30eafae6-b515-4f43-b776-457bd9a323de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:34:52.899394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:52.899394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:52.899394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:52.899394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:52.899394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:52.899394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:52.899394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:52.899394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:52.899394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:52.899394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:52.899394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:52.899394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:34:52.899394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:34:52.899394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:34:52.899394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:34:52.900548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:34:52.900548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:34:52.900548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:34:52.900548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:34:52.900548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:52.900548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:52.900548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:52.900548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:52.900548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:52.900548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:52.900548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:52.900548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:52.900548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:52.900548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:52.900548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:52.900548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:52.901637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:52.901637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:52.901637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:52.901637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:52.901637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:34:52.901637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:34:52.901637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:34:52.901637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:34:52.901637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:34:52.901637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:34:52.901637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:34:52.901637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:52.904913 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 62/136] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:34:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:34:52.905251 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:52.912136 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c97e8b-7e30-4861-a910-4536d935d97e None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:52.984536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:53.023258 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c97e8b-7e30-4861-a910-4536d935d97e None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:53.068369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914f5f1-f02c-4eb7-a8e3-1a8597df4bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:53.069549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914f5f1-f02c-4eb7-a8e3-1a8597df4bbe None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:53.069990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914f5f1-f02c-4eb7-a8e3-1a8597df4bbe None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:53.070266 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914f5f1-f02c-4eb7-a8e3-1a8597df4bbe None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:53.070956 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914f5f1-f02c-4eb7-a8e3-1a8597df4bbe None admin] RBAC: Adding query filter params (name=reader) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:53.071264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914f5f1-f02c-4eb7-a8e3-1a8597df4bbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:53.072741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914f5f1-f02c-4eb7-a8e3-1a8597df4bbe None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:53.082704 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 75/137] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:34:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:53.088588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eafae6-b515-4f43-b776-457bd9a323de None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:53.203255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eafae6-b515-4f43-b776-457bd9a323de None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:53.246145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af4193-1c2e-4bf6-9433-15e7f3b55b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:53.246720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af4193-1c2e-4bf6-9433-15e7f3b55b61 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:53.247008 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af4193-1c2e-4bf6-9433-15e7f3b55b61 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:53.247287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af4193-1c2e-4bf6-9433-15e7f3b55b61 None admin] PATH_INFO: `/v3/users/fded43439072492ea7816449fe30d3bf` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:53.248483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af4193-1c2e-4bf6-9433-15e7f3b55b61 None admin] RBAC: Authorizing `identity:get_user(user_id=fded43439072492ea7816449fe30d3bf)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:53.250293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af4193-1c2e-4bf6-9433-15e7f3b55b61 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:53.252275 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 63/138] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:34:53 2022] GET /identity/v3/users/fded43439072492ea7816449fe30d3bf => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:53.257786 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914f5f1-f02c-4eb7-a8e3-1a8597df4bbe None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:53.386792 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914f5f1-f02c-4eb7-a8e3-1a8597df4bbe None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:53.420038 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:53.427388 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82646164-66b4-4114-9a1c-d130063be0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:53.428014 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82646164-66b4-4114-9a1c-d130063be0a8 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:53.428315 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82646164-66b4-4114-9a1c-d130063be0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:53.429115 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82646164-66b4-4114-9a1c-d130063be0a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:53.505865 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82646164-66b4-4114-9a1c-d130063be0a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:53.508457 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82646164-66b4-4114-9a1c-d130063be0a8 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:53.625670 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 76/139] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:34:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:34:53.631256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af4193-1c2e-4bf6-9433-15e7f3b55b61 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:53.716384 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:53.748398 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af4193-1c2e-4bf6-9433-15e7f3b55b61 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:53.785296 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c8401-1bcc-4614-9af8-89af835fc9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:53.785871 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c8401-1bcc-4614-9af8-89af835fc9f8 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:53.786487 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c8401-1bcc-4614-9af8-89af835fc9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:53.786773 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c8401-1bcc-4614-9af8-89af835fc9f8 None admin] PATH_INFO: `/v3/projects/72d02777c6694301959a12c02452adf3` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:53.788493 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c8401-1bcc-4614-9af8-89af835fc9f8 None admin] RBAC: Authorizing `identity:get_project(project_id=72d02777c6694301959a12c02452adf3)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:53.790073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c8401-1bcc-4614-9af8-89af835fc9f8 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:53.791927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 64/140] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:34:53 2022] GET /identity/v3/projects/72d02777c6694301959a12c02452adf3 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:53.797774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82646164-66b4-4114-9a1c-d130063be0a8 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:53.834029 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:53.921823 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82646164-66b4-4114-9a1c-d130063be0a8 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:53.958747 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc32d1-3dce-4020-a092-7c96158ba64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:53.960330 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc32d1-3dce-4020-a092-7c96158ba64e None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:53.961016 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc32d1-3dce-4020-a092-7c96158ba64e None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:53.961495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc32d1-3dce-4020-a092-7c96158ba64e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:53.962425 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc32d1-3dce-4020-a092-7c96158ba64e None admin] RBAC: Adding query filter params (role.id=2e58f1f4fb154236982461a05ee16601, scope.project.id=72d02777c6694301959a12c02452adf3, user.id=fded43439072492ea7816449fe30d3bf) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:53.962964 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc32d1-3dce-4020-a092-7c96158ba64e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:53.964664 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc32d1-3dce-4020-a092-7c96158ba64e None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:53.973249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 77/141] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:34:53 2022] GET /identity/v3/role_assignments?user.id=fded43439072492ea7816449fe30d3bf&scope.project.id=72d02777c6694301959a12c02452adf3&role.id=2e58f1f4fb154236982461a05ee16601 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:54.488751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdc088-f460-404b-985f-63c32f7b01f0 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:54.489124 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdc088-f460-404b-985f-63c32f7b01f0 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:54.489331 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdc088-f460-404b-985f-63c32f7b01f0 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:54.490693 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 65/142] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:54.496498 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b543395-352e-432f-acda-e82055fb9f28 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:54.497013 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b543395-352e-432f-acda-e82055fb9f28 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:54.497088 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b543395-352e-432f-acda-e82055fb9f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:54.517274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-3b543395-352e-432f-acda-e82055fb9f28 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:54.518501 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-3b543395-352e-432f-acda-e82055fb9f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:54.598459 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 78/143] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:54.675784 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327f316-8948-46cd-a256-1915ce9b0b79 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:54.676178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327f316-8948-46cd-a256-1915ce9b0b79 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:54.676534 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327f316-8948-46cd-a256-1915ce9b0b79 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:54.677992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 66/144] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:54.685688 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b14169-f91e-4a23-924c-08bd6409bd7f None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:54.685688 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b14169-f91e-4a23-924c-08bd6409bd7f None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:54.685688 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b14169-f91e-4a23-924c-08bd6409bd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:54.703981 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-c6b14169-f91e-4a23-924c-08bd6409bd7f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:54.705206 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c6b14169-f91e-4a23-924c-08bd6409bd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:54.727382 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:54.749766 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd164ee4-b420-4f4a-a79f-2ebcfab87a06 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:54.750270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd164ee4-b420-4f4a-a79f-2ebcfab87a06 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:54.750270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd164ee4-b420-4f4a-a79f-2ebcfab87a06 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:54.751721 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 67/145] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:34:54.757762 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cedc14-03d0-4cdd-9b11-cdd04db6bccd None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:54.758086 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cedc14-03d0-4cdd-9b11-cdd04db6bccd None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:54.758363 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cedc14-03d0-4cdd-9b11-cdd04db6bccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:54.776878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-e9cedc14-03d0-4cdd-9b11-cdd04db6bccd None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:54.778098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e9cedc14-03d0-4cdd-9b11-cdd04db6bccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:54.785770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 79/146] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:54.854303 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 68/147] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:34:54.878056 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b14169-f91e-4a23-924c-08bd6409bd7f None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:54.992276 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b14169-f91e-4a23-924c-08bd6409bd7f None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:55.029798 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e50316-5d12-4060-a365-21e59c2a7452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:55.030358 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e50316-5d12-4060-a365-21e59c2a7452 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:55.030640 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e50316-5d12-4060-a365-21e59c2a7452 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:55.030916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e50316-5d12-4060-a365-21e59c2a7452 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:55.042400 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e50316-5d12-4060-a365-21e59c2a7452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:34:55.047456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-69e50316-5d12-4060-a365-21e59c2a7452 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:34:55.047456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:55.047456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:55.047456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:55.047456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:55.047456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:55.047456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:55.047456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:55.047456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:55.047456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:55.047456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:55.047456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:55.047456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:34:55.047456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:34:55.047456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:34:55.047456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:34:55.048373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:34:55.048373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:34:55.048373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:34:55.048373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:34:55.048373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:55.048373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:55.048373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:55.048373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:55.048373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:55.048373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:55.048373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:55.048373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:55.048373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:55.048373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:55.048373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:55.048373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:55.049412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:55.049412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:55.049412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:55.049412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:55.049412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:34:55.049412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:34:55.049412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:34:55.049412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:34:55.049412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:34:55.049412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:34:55.049412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:34:55.049412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:34:55.049412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 80/148] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:34:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:34:55.056943 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cedc14-03d0-4cdd-9b11-cdd04db6bccd None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:55.059049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e50316-5d12-4060-a365-21e59c2a7452 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:55.082235 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:55.140002 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:55.174624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e50316-5d12-4060-a365-21e59c2a7452 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:55.179057 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cedc14-03d0-4cdd-9b11-cdd04db6bccd None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:55.214236 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25292ef5-5672-4736-9932-6c4e534d839d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:55.214872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25292ef5-5672-4736-9932-6c4e534d839d None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:55.215154 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25292ef5-5672-4736-9932-6c4e534d839d None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:55.215433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25292ef5-5672-4736-9932-6c4e534d839d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:55.222814 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7da750-8369-4c8d-84f6-c0de30c1d5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:55.223864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7da750-8369-4c8d-84f6-c0de30c1d5c4 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:55.224272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7da750-8369-4c8d-84f6-c0de30c1d5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:55.224644 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7da750-8369-4c8d-84f6-c0de30c1d5c4 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:55.225478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7da750-8369-4c8d-84f6-c0de30c1d5c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:55.225955 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7da750-8369-4c8d-84f6-c0de30c1d5c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:55.226437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25292ef5-5672-4736-9932-6c4e534d839d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:34:55.227654 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7da750-8369-4c8d-84f6-c0de30c1d5c4 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:55.231375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-25292ef5-5672-4736-9932-6c4e534d839d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:34:55.231375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:55.231375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:55.231375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:55.231375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:55.231375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:55.231375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:55.231375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:55.231375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:55.231375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:55.231375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:55.231375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:55.231375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:34:55.231375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:34:55.231375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:34:55.231375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:34:55.233305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:34:55.233305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:34:55.233305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:34:55.233305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:34:55.233305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:55.233305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:55.233305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:55.233305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:55.233305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:55.233305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:55.233305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:55.233305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:55.233305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:55.233305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:55.233305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:55.233305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:55.235181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:55.235181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:55.235181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:55.235181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:55.235181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:34:55.235181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:34:55.235181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:34:55.235181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:34:55.235181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:34:55.235181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:34:55.235181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:34:55.235181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:34:55.237380 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 81/149] 10.222.0.36 () {56 vars in 1114 bytes} [Sun Oct 30 00:34:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:34:55.238277 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25292ef5-5672-4736-9932-6c4e534d839d None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:55.239947 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 69/150] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:34:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:55.241034 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7da750-8369-4c8d-84f6-c0de30c1d5c4 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:55.328955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:55.353412 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7da750-8369-4c8d-84f6-c0de30c1d5c4 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:55.356542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25292ef5-5672-4736-9932-6c4e534d839d None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:55.388602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:55.394383 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14082b6f-7c36-42f0-9156-526405314bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:55.395027 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14082b6f-7c36-42f0-9156-526405314bf3 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:55.395319 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14082b6f-7c36-42f0-9156-526405314bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:55.395581 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14082b6f-7c36-42f0-9156-526405314bf3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:55.396701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14082b6f-7c36-42f0-9156-526405314bf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:55.398986 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14082b6f-7c36-42f0-9156-526405314bf3 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:55.399332 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c57bc1-9907-4ab5-8459-53cdc077bb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:55.399957 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c57bc1-9907-4ab5-8459-53cdc077bb00 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:55.400269 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c57bc1-9907-4ab5-8459-53cdc077bb00 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:55.400551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c57bc1-9907-4ab5-8459-53cdc077bb00 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:55.400943 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 82/151] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:34:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:55.401229 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c57bc1-9907-4ab5-8459-53cdc077bb00 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:55.401536 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c57bc1-9907-4ab5-8459-53cdc077bb00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:55.401715 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14082b6f-7c36-42f0-9156-526405314bf3 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:55.403009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c57bc1-9907-4ab5-8459-53cdc077bb00 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:55.420052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 70/152] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:34:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:55.420052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c57bc1-9907-4ab5-8459-53cdc077bb00 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:55.512751 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14082b6f-7c36-42f0-9156-526405314bf3 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:55.525224 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c57bc1-9907-4ab5-8459-53cdc077bb00 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:55.548936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ddab4-0c83-42d3-9e6c-28885d1284e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:55.549580 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ddab4-0c83-42d3-9e6c-28885d1284e7 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:55.549862 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ddab4-0c83-42d3-9e6c-28885d1284e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:55.550133 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ddab4-0c83-42d3-9e6c-28885d1284e7 None admin] PATH_INFO: `/v3/users/c43f4a296b7a4b0f902099eff4bb3939` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:55.562758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:55.562943 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b056aef-3468-4669-9c05-424d889039a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:55.563600 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b056aef-3468-4669-9c05-424d889039a9 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:55.563913 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b056aef-3468-4669-9c05-424d889039a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:55.564204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b056aef-3468-4669-9c05-424d889039a9 None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:55.564925 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b056aef-3468-4669-9c05-424d889039a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:55.565358 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ddab4-0c83-42d3-9e6c-28885d1284e7 None admin] RBAC: Authorizing `identity:get_user(user_id=c43f4a296b7a4b0f902099eff4bb3939)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:55.566440 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b056aef-3468-4669-9c05-424d889039a9 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:55.566844 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ddab4-0c83-42d3-9e6c-28885d1284e7 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:55.568765 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 83/153] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:34:55 2022] GET /identity/v3/users/c43f4a296b7a4b0f902099eff4bb3939 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:55.570348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ddab4-0c83-42d3-9e6c-28885d1284e7 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:55.612214 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 71/154] 10.222.0.36 () {60 vars in 1132 bytes} [Sun Oct 30 00:34:55 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:34:55.612214 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b056aef-3468-4669-9c05-424d889039a9 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:55.657333 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:55.695303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ddab4-0c83-42d3-9e6c-28885d1284e7 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:55.731836 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b056aef-3468-4669-9c05-424d889039a9 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:55.739943 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee08710-0332-4676-9548-c661e4867347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:55.740652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee08710-0332-4676-9548-c661e4867347 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:55.740936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee08710-0332-4676-9548-c661e4867347 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:55.741289 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee08710-0332-4676-9548-c661e4867347 None admin] PATH_INFO: `/v3/users/c43f4a296b7a4b0f902099eff4bb3939` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:55.743051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee08710-0332-4676-9548-c661e4867347 None admin] RBAC: Authorizing `identity:get_user(user_id=c43f4a296b7a4b0f902099eff4bb3939)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:55.745141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee08710-0332-4676-9548-c661e4867347 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:55.751633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 84/155] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:34:55 2022] GET /identity/v3/users/c43f4a296b7a4b0f902099eff4bb3939 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:55.752708 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee08710-0332-4676-9548-c661e4867347 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:55.783447 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25316150-6c47-49a8-80c8-bc224c3637c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:55.784784 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25316150-6c47-49a8-80c8-bc224c3637c9 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:55.784784 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25316150-6c47-49a8-80c8-bc224c3637c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:55.784784 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25316150-6c47-49a8-80c8-bc224c3637c9 None admin] PATH_INFO: `/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:55.786470 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25316150-6c47-49a8-80c8-bc224c3637c9 None admin] RBAC: Authorizing `identity:get_project(project_id=b9c6b20f7b7a4967ada2d4c9d25dd94a)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:55.789634 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25316150-6c47-49a8-80c8-bc224c3637c9 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:55.790684 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 72/156] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:34:55 2022] GET /identity/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:55.797724 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25316150-6c47-49a8-80c8-bc224c3637c9 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:55.837636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:55.883472 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee08710-0332-4676-9548-c661e4867347 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:55.932513 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f0ec6-8b59-4f41-80bd-14702b2f13a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:55.933179 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f0ec6-8b59-4f41-80bd-14702b2f13a7 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:55.933461 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f0ec6-8b59-4f41-80bd-14702b2f13a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:55.933737 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f0ec6-8b59-4f41-80bd-14702b2f13a7 None admin] PATH_INFO: `/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:55.935382 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f0ec6-8b59-4f41-80bd-14702b2f13a7 None admin] RBAC: Authorizing `identity:get_project(project_id=b9c6b20f7b7a4967ada2d4c9d25dd94a)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:55.936933 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f0ec6-8b59-4f41-80bd-14702b2f13a7 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:55.938959 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 85/157] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:34:55 2022] GET /identity/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:55.943454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25316150-6c47-49a8-80c8-bc224c3637c9 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:55.948371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f0ec6-8b59-4f41-80bd-14702b2f13a7 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:55.961796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:55.996474 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab3f1d-5026-4613-8afa-a1abd3bd3991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:55.997324 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab3f1d-5026-4613-8afa-a1abd3bd3991 None admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:55.997446 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab3f1d-5026-4613-8afa-a1abd3bd3991 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:55.997700 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab3f1d-5026-4613-8afa-a1abd3bd3991 None admin] PATH_INFO: `/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a/users/c43f4a296b7a4b0f902099eff4bb3939/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:55.999812 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab3f1d-5026-4613-8afa-a1abd3bd3991 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9c6b20f7b7a4967ada2d4c9d25dd94a, user_id=c43f4a296b7a4b0f902099eff4bb3939, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:56.001394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab3f1d-5026-4613-8afa-a1abd3bd3991 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:56.020315 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 73/158] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:34:55 2022] PUT /identity/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a/users/c43f4a296b7a4b0f902099eff4bb3939/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:34:56.095662 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f0ec6-8b59-4f41-80bd-14702b2f13a7 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:56.144044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a9d01-65be-4edf-bef9-b97e4635a6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:56.144844 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a9d01-65be-4edf-bef9-b97e4635a6b3 None admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:56.145129 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a9d01-65be-4edf-bef9-b97e4635a6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:56.145409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a9d01-65be-4edf-bef9-b97e4635a6b3 None admin] PATH_INFO: `/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a/users/c43f4a296b7a4b0f902099eff4bb3939/roles/af3529d63bae4650be7b8dc67b3ed527` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:56.147903 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a9d01-65be-4edf-bef9-b97e4635a6b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9c6b20f7b7a4967ada2d4c9d25dd94a, user_id=c43f4a296b7a4b0f902099eff4bb3939, role_id=af3529d63bae4650be7b8dc67b3ed527)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:56.149446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a9d01-65be-4edf-bef9-b97e4635a6b3 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:56.153070 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:56.170311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 86/159] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:34:55 2022] PUT /identity/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a/users/c43f4a296b7a4b0f902099eff4bb3939/roles/af3529d63bae4650be7b8dc67b3ed527 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:34:57.941994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c07f83-4703-4458-91a0-99aacd85f9ea None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:57.942544 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c07f83-4703-4458-91a0-99aacd85f9ea None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:57.942544 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c07f83-4703-4458-91a0-99aacd85f9ea None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:57.944213 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 74/160] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:57.951760 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34332e-769a-40eb-a60c-0a67b0db6ba2 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:57.952782 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34332e-769a-40eb-a60c-0a67b0db6ba2 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:57.953258 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34332e-769a-40eb-a60c-0a67b0db6ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:57.972882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-2c34332e-769a-40eb-a60c-0a67b0db6ba2 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:57.974447 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-2c34332e-769a-40eb-a60c-0a67b0db6ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:58.055298 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 87/161] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:58.169568 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96956da-47df-4738-879f-744ade8e21d1 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:58.169945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96956da-47df-4738-879f-744ade8e21d1 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:58.170113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96956da-47df-4738-879f-744ade8e21d1 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:58.171715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 75/162] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:58.177248 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ba04-6f42-4564-9cd2-e9ac3274e063 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:58.177531 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ba04-6f42-4564-9cd2-e9ac3274e063 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:58.177798 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ba04-6f42-4564-9cd2-e9ac3274e063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:58.200216 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-f6c9ba04-6f42-4564-9cd2-e9ac3274e063 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:58.201856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-f6c9ba04-6f42-4564-9cd2-e9ac3274e063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:58.284722 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 88/163] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:58.324717 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a1bbe-fd1e-4cc4-8ce6-a8126bdcb524 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:58.325076 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a1bbe-fd1e-4cc4-8ce6-a8126bdcb524 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:58.325271 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a1bbe-fd1e-4cc4-8ce6-a8126bdcb524 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:58.326647 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 76/164] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:58.332232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a1bbe-fd1e-4cc4-8ce6-a8126bdcb524 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:58.333017 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c27c1c-037d-45cd-ae02-b89a9b27d2e1 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:58.333310 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c27c1c-037d-45cd-ae02-b89a9b27d2e1 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:58.333586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c27c1c-037d-45cd-ae02-b89a9b27d2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:58.352570 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-a6c27c1c-037d-45cd-ae02-b89a9b27d2e1 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:58.353772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a6c27c1c-037d-45cd-ae02-b89a9b27d2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:58.418199 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:58.438378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 89/165] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:58.471394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a1bbe-fd1e-4cc4-8ce6-a8126bdcb524 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:58.507632 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159be29-0be4-4cf6-af0c-b6a2e8d72c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:58.508377 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159be29-0be4-4cf6-af0c-b6a2e8d72c71 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:58.508707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159be29-0be4-4cf6-af0c-b6a2e8d72c71 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:58.508995 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159be29-0be4-4cf6-af0c-b6a2e8d72c71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:58.519972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b159be29-0be4-4cf6-af0c-b6a2e8d72c71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:34:58.526369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-b159be29-0be4-4cf6-af0c-b6a2e8d72c71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:34:58.526369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:58.526369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:58.526369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:58.526369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:58.526369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:58.526369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:58.526369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:58.526369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:58.526369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:58.526369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:58.526369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:58.526369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:34:58.526369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:34:58.526369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:34:58.526369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:34:58.527478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:34:58.527478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:34:58.527478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:34:58.527478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:34:58.527478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:58.527478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:58.527478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:58.527478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:58.527478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:58.527478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:58.527478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:58.527478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:58.527478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:58.527478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:58.527478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:58.527478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:58.528467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:58.528467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:58.528467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:58.528467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:58.528467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:34:58.528467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:34:58.528467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:34:58.528467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:34:58.528467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:34:58.528467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:34:58.528467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:34:58.528467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:58.532129 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 77/166] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:34:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:34:58.533842 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c27c1c-037d-45cd-ae02-b89a9b27d2e1 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:58.585901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159be29-0be4-4cf6-af0c-b6a2e8d72c71 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:58.642749 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:58.662516 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c27c1c-037d-45cd-ae02-b89a9b27d2e1 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:58.701152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10aa30d-eb6a-461e-a68f-5bad270db898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:58.701690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aa30d-eb6a-461e-a68f-5bad270db898 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:58.701968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aa30d-eb6a-461e-a68f-5bad270db898 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:58.702237 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aa30d-eb6a-461e-a68f-5bad270db898 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:58.702901 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10aa30d-eb6a-461e-a68f-5bad270db898 None admin] RBAC: Adding query filter params (name=member) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:58.703196 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10aa30d-eb6a-461e-a68f-5bad270db898 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:58.704741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10aa30d-eb6a-461e-a68f-5bad270db898 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:58.714011 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 90/167] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:34:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:58.719588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10aa30d-eb6a-461e-a68f-5bad270db898 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:58.748774 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159be29-0be4-4cf6-af0c-b6a2e8d72c71 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:58.774745 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:58.785973 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76320cca-2390-4a19-8211-b954d70c99b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:58.786945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76320cca-2390-4a19-8211-b954d70c99b3 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:58.787219 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76320cca-2390-4a19-8211-b954d70c99b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:58.787490 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76320cca-2390-4a19-8211-b954d70c99b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:58.789856 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76320cca-2390-4a19-8211-b954d70c99b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:58.791297 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76320cca-2390-4a19-8211-b954d70c99b3 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:58.797535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 78/168] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:34:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:58.798392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76320cca-2390-4a19-8211-b954d70c99b3 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:58.832227 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10aa30d-eb6a-461e-a68f-5bad270db898 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:58.876670 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2de4a-8a36-4296-9215-ebd6f87e8981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:58.877348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2de4a-8a36-4296-9215-ebd6f87e8981 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:58.877632 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2de4a-8a36-4296-9215-ebd6f87e8981 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:58.877907 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2de4a-8a36-4296-9215-ebd6f87e8981 None admin] PATH_INFO: `/v3/users/f49944cb82fb436cab591a97ba89b2e6` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:58.895135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2de4a-8a36-4296-9215-ebd6f87e8981 None admin] RBAC: Authorizing `identity:get_user(user_id=f49944cb82fb436cab591a97ba89b2e6)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:58.896906 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2de4a-8a36-4296-9215-ebd6f87e8981 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:58.898991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 91/169] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:34:58 2022] GET /identity/v3/users/f49944cb82fb436cab591a97ba89b2e6 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:34:58.899646 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2de4a-8a36-4296-9215-ebd6f87e8981 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:58.921953 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:58.926525 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76320cca-2390-4a19-8211-b954d70c99b3 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:58.965870 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6d533-5051-4fc6-8917-0c4ac2baa83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:58.966492 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6d533-5051-4fc6-8917-0c4ac2baa83f None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:58.966699 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6d533-5051-4fc6-8917-0c4ac2baa83f None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:58.966981 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6d533-5051-4fc6-8917-0c4ac2baa83f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:58.977399 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6d533-5051-4fc6-8917-0c4ac2baa83f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:34:58.982218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-65a6d533-5051-4fc6-8917-0c4ac2baa83f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:34:58.982218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:58.982218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:58.982218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:58.982218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:58.982218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:58.982218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:58.982218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:58.982218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:58.982218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:58.982218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:58.982218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:58.982218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:34:58.982218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:34:58.982218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:34:58.982218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:34:58.983077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:34:58.983077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:34:58.983077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:34:58.983077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:34:58.983077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:58.983077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:58.983077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:58.983077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:58.983077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:58.983077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:58.983077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:58.983077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:58.983077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:58.983077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:58.983077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:58.983077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:58.983901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:58.983901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:58.983901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:58.983901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:58.983901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:34:58.983901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:34:58.983901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:34:58.983901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:34:58.983901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:34:58.983901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:34:58.983901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:34:58.983901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:34:58.983901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 79/170] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:34:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:34:58.985194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6d533-5051-4fc6-8917-0c4ac2baa83f None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:59.014641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2de4a-8a36-4296-9215-ebd6f87e8981 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:59.022326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:59.056845 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca9b30-32d7-4202-a8d9-586875712dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:59.057820 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca9b30-32d7-4202-a8d9-586875712dc5 None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:59.058299 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca9b30-32d7-4202-a8d9-586875712dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:59.058765 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca9b30-32d7-4202-a8d9-586875712dc5 None admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:59.059877 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ca9b30-32d7-4202-a8d9-586875712dc5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:59.062515 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ca9b30-32d7-4202-a8d9-586875712dc5 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:59.090182 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 92/171] 10.222.0.36 () {60 vars in 1132 bytes} [Sun Oct 30 00:34:58 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:34:59.092497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6d533-5051-4fc6-8917-0c4ac2baa83f None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:59.092675 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b962ca1-44e6-44f9-a319-64caead55a5e None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:59.092675 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b962ca1-44e6-44f9-a319-64caead55a5e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:59.092675 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b962ca1-44e6-44f9-a319-64caead55a5e None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:59.093517 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 93/172] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:34:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:34:59.093959 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b962ca1-44e6-44f9-a319-64caead55a5e None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:59.139486 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef037195-c896-4565-9d22-39bb1f716917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:59.140163 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef037195-c896-4565-9d22-39bb1f716917 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:59.140620 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef037195-c896-4565-9d22-39bb1f716917 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:59.140892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef037195-c896-4565-9d22-39bb1f716917 None admin] PATH_INFO: `/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:59.142485 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef037195-c896-4565-9d22-39bb1f716917 None admin] RBAC: Authorizing `identity:get_project(project_id=b9c6b20f7b7a4967ada2d4c9d25dd94a)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:59.143979 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef037195-c896-4565-9d22-39bb1f716917 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:59.145757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 80/173] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:34:58 2022] GET /identity/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:59.152643 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa54f8-3b78-4464-b554-ac5f1fb63a84 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:59.153018 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa54f8-3b78-4464-b554-ac5f1fb63a84 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:59.153197 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa54f8-3b78-4464-b554-ac5f1fb63a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:59.173195 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-32aa54f8-3b78-4464-b554-ac5f1fb63a84 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:34:59.174434 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-32aa54f8-3b78-4464-b554-ac5f1fb63a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:34:59.200585 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:59.226319 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b962ca1-44e6-44f9-a319-64caead55a5e None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:59.254553 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 81/174] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:34:59.255411 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa54f8-3b78-4464-b554-ac5f1fb63a84 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:59.286454 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3cbf7-d7ad-43eb-94a3-dc679f5b8230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:59.287347 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3cbf7-d7ad-43eb-94a3-dc679f5b8230 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:59.287782 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3cbf7-d7ad-43eb-94a3-dc679f5b8230 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:59.288237 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3cbf7-d7ad-43eb-94a3-dc679f5b8230 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:59.288893 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3cbf7-d7ad-43eb-94a3-dc679f5b8230 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:59.289294 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3cbf7-d7ad-43eb-94a3-dc679f5b8230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:59.290706 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3cbf7-d7ad-43eb-94a3-dc679f5b8230 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:59.299400 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 94/175] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:34:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:59.305675 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3cbf7-d7ad-43eb-94a3-dc679f5b8230 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:59.334499 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:59.399654 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa54f8-3b78-4464-b554-ac5f1fb63a84 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:59.450493 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3cbf7-d7ad-43eb-94a3-dc679f5b8230 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:59.451256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4166e-4c29-4635-8f3d-ba44ac5fca4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:59.451894 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4166e-4c29-4635-8f3d-ba44ac5fca4e None admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:59.452503 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4166e-4c29-4635-8f3d-ba44ac5fca4e None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:59.452690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4166e-4c29-4635-8f3d-ba44ac5fca4e None admin] PATH_INFO: `/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a/users/f49944cb82fb436cab591a97ba89b2e6/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:59.454600 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4166e-4c29-4635-8f3d-ba44ac5fca4e None admin] RBAC: Authorizing `identity:create_grant(project_id=b9c6b20f7b7a4967ada2d4c9d25dd94a, user_id=f49944cb82fb436cab591a97ba89b2e6, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:59.456057 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4166e-4c29-4635-8f3d-ba44ac5fca4e None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:59.473809 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 82/176] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:34:59 2022] PUT /identity/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a/users/f49944cb82fb436cab591a97ba89b2e6/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:34:59.497169 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c2bc3-597c-407f-bb8e-c4ab1ac13f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:59.497860 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c2bc3-597c-407f-bb8e-c4ab1ac13f8a None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:59.498207 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c2bc3-597c-407f-bb8e-c4ab1ac13f8a None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:59.498544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c2bc3-597c-407f-bb8e-c4ab1ac13f8a None admin] PATH_INFO: `/v3/users/c43f4a296b7a4b0f902099eff4bb3939` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:59.499879 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c2bc3-597c-407f-bb8e-c4ab1ac13f8a None admin] RBAC: Authorizing `identity:get_user(user_id=c43f4a296b7a4b0f902099eff4bb3939)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:59.501436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c2bc3-597c-407f-bb8e-c4ab1ac13f8a None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:59.503380 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 95/177] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:34:59 2022] GET /identity/v3/users/c43f4a296b7a4b0f902099eff4bb3939 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:34:59.509963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4166e-4c29-4635-8f3d-ba44ac5fca4e None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:59.536659 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:59.551778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c2bc3-597c-407f-bb8e-c4ab1ac13f8a None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:59.643704 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4166e-4c29-4635-8f3d-ba44ac5fca4e None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:59.683451 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b2ca0-6ecd-468e-bc45-7980c43b51a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:59.683932 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:59.684521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b2ca0-6ecd-468e-bc45-7980c43b51a3 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:59.684933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b2ca0-6ecd-468e-bc45-7980c43b51a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:59.685399 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b2ca0-6ecd-468e-bc45-7980c43b51a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:59.696693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c2bc3-597c-407f-bb8e-c4ab1ac13f8a None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:59.747870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcacdf-ee93-4d1a-8e28-fbe09ecef86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:59.749186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcacdf-ee93-4d1a-8e28-fbe09ecef86a None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:59.749547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcacdf-ee93-4d1a-8e28-fbe09ecef86a None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:59.749832 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcacdf-ee93-4d1a-8e28-fbe09ecef86a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:59.766872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fcacdf-ee93-4d1a-8e28-fbe09ecef86a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:34:59.774730 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b2ca0-6ecd-468e-bc45-7980c43b51a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:59.774902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-46fcacdf-ee93-4d1a-8e28-fbe09ecef86a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:34:59.774902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:34:59.774902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:34:59.774902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:34:59.774902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:34:59.774902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:34:59.774902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:34:59.774902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:34:59.774902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:34:59.774902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:34:59.774902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:34:59.774902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:34:59.774902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:34:59.774902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:34:59.774902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:34:59.774902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:34:59.775923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:34:59.775923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:34:59.775923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:34:59.775923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:34:59.775923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:34:59.775923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:34:59.775923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:34:59.775923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:34:59.775923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:34:59.775923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:34:59.775923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:34:59.775923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:34:59.775923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:34:59.775923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:34:59.775923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:34:59.775923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:34:59.776873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:34:59.776873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:34:59.776873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:34:59.776873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:34:59.776873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:34:59.776873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:34:59.776873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:34:59.776873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:34:59.776873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:34:59.776873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:34:59.776873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:34:59.776873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:34:59.776873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 96/178] 10.222.0.36 () {56 vars in 1114 bytes} [Sun Oct 30 00:34:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:34:59.779246 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b2ca0-6ecd-468e-bc45-7980c43b51a3 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:59.783253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcacdf-ee93-4d1a-8e28-fbe09ecef86a None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:34:59.817591 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:34:59.919191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcacdf-ee93-4d1a-8e28-fbe09ecef86a None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:34:59.975094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed3f9d-a6dc-45bd-9ac5-a1be92340228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:34:59.975793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed3f9d-a6dc-45bd-9ac5-a1be92340228 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:34:59.976121 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed3f9d-a6dc-45bd-9ac5-a1be92340228 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:34:59.976365 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed3f9d-a6dc-45bd-9ac5-a1be92340228 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:34:59.977041 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed3f9d-a6dc-45bd-9ac5-a1be92340228 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:34:59.977343 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed3f9d-a6dc-45bd-9ac5-a1be92340228 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:34:59.978761 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed3f9d-a6dc-45bd-9ac5-a1be92340228 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:34:59.979627 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 83/179] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:34:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:34:59.992618 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 97/180] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:34:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:34:59.993883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b2ca0-6ecd-468e-bc45-7980c43b51a3 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:00.010689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed3f9d-a6dc-45bd-9ac5-a1be92340228 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:00.057294 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:00.116970 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b2ca0-6ecd-468e-bc45-7980c43b51a3 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:00.143884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:00.168872 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1aeea-b142-447b-af0f-dbcebdb6af82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:00.170022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1aeea-b142-447b-af0f-dbcebdb6af82 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:00.170454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1aeea-b142-447b-af0f-dbcebdb6af82 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:00.170733 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1aeea-b142-447b-af0f-dbcebdb6af82 None admin] PATH_INFO: `/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:00.171895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1aeea-b142-447b-af0f-dbcebdb6af82 None admin] RBAC: Authorizing `identity:get_project(project_id=b9c6b20f7b7a4967ada2d4c9d25dd94a)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:00.173400 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1aeea-b142-447b-af0f-dbcebdb6af82 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:00.175129 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 84/181] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:34:59 2022] GET /identity/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:00.181313 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1aeea-b142-447b-af0f-dbcebdb6af82 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:00.265284 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed3f9d-a6dc-45bd-9ac5-a1be92340228 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:00.337757 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ae00b-8945-4802-b18f-210f0745a8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:00.338414 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ae00b-8945-4802-b18f-210f0745a8e2 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:00.338698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ae00b-8945-4802-b18f-210f0745a8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:00.338978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ae00b-8945-4802-b18f-210f0745a8e2 None admin] PATH_INFO: `/v3/users/c43f4a296b7a4b0f902099eff4bb3939` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:00.340305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ae00b-8945-4802-b18f-210f0745a8e2 None admin] RBAC: Authorizing `identity:get_user(user_id=c43f4a296b7a4b0f902099eff4bb3939)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:00.343266 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ae00b-8945-4802-b18f-210f0745a8e2 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:00.344927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 98/182] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:35:00 2022] GET /identity/v3/users/c43f4a296b7a4b0f902099eff4bb3939 => generated 283 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:00.352112 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ae00b-8945-4802-b18f-210f0745a8e2 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:00.364001 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1aeea-b142-447b-af0f-dbcebdb6af82 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:00.410080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b78ce7-92cf-4656-837d-04458c34f719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:00.411033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b78ce7-92cf-4656-837d-04458c34f719 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:00.411601 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b78ce7-92cf-4656-837d-04458c34f719 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:00.411883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b78ce7-92cf-4656-837d-04458c34f719 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:00.412730 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b78ce7-92cf-4656-837d-04458c34f719 None admin] RBAC: Adding query filter params (role.id=980241d45fdb435d9f32c434057c71e4, scope.project.id=b9c6b20f7b7a4967ada2d4c9d25dd94a, user.id=c43f4a296b7a4b0f902099eff4bb3939) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:00.413047 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b78ce7-92cf-4656-837d-04458c34f719 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:00.415515 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b78ce7-92cf-4656-837d-04458c34f719 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:00.424220 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 85/183] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:35:00 2022] GET /identity/v3/role_assignments?user.id=c43f4a296b7a4b0f902099eff4bb3939&scope.project.id=b9c6b20f7b7a4967ada2d4c9d25dd94a&role.id=980241d45fdb435d9f32c434057c71e4 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:00.526622 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:00.553590 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ae00b-8945-4802-b18f-210f0745a8e2 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:00.627922 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a39ca4-0772-43bf-911d-d44581f7b624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:00.628604 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a39ca4-0772-43bf-911d-d44581f7b624 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:00.628888 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a39ca4-0772-43bf-911d-d44581f7b624 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:00.629162 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a39ca4-0772-43bf-911d-d44581f7b624 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:00.729219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a39ca4-0772-43bf-911d-d44581f7b624 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:00.730703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a39ca4-0772-43bf-911d-d44581f7b624 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:00.821918 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:00.843778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 99/184] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:35:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:35:00.849258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b78ce7-92cf-4656-837d-04458c34f719 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:00.930932 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:00.988642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b78ce7-92cf-4656-837d-04458c34f719 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:01.028550 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f762766-b6ab-4134-8a0e-f7422ad7ec86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:01.029246 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f762766-b6ab-4134-8a0e-f7422ad7ec86 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:01.029533 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f762766-b6ab-4134-8a0e-f7422ad7ec86 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:01.029811 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f762766-b6ab-4134-8a0e-f7422ad7ec86 None admin] PATH_INFO: `/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:01.030956 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f762766-b6ab-4134-8a0e-f7422ad7ec86 None admin] RBAC: Authorizing `identity:get_project(project_id=b9c6b20f7b7a4967ada2d4c9d25dd94a)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:01.032451 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f762766-b6ab-4134-8a0e-f7422ad7ec86 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:01.034109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 86/185] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:35:00 2022] GET /identity/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:01.041486 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a39ca4-0772-43bf-911d-d44581f7b624 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:01.156604 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a39ca4-0772-43bf-911d-d44581f7b624 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:01.199470 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca1944-2705-441e-9625-a0896c165382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:01.200102 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1944-2705-441e-9625-a0896c165382 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:01.200397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1944-2705-441e-9625-a0896c165382 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:01.200678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca1944-2705-441e-9625-a0896c165382 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:01.201410 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca1944-2705-441e-9625-a0896c165382 None admin] RBAC: Adding query filter params (role.id=af3529d63bae4650be7b8dc67b3ed527, scope.project.id=b9c6b20f7b7a4967ada2d4c9d25dd94a, user.id=c43f4a296b7a4b0f902099eff4bb3939) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:01.201718 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca1944-2705-441e-9625-a0896c165382 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:01.203211 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca1944-2705-441e-9625-a0896c165382 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:01.209644 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 100/186] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:35:01 2022] GET /identity/v3/role_assignments?user.id=c43f4a296b7a4b0f902099eff4bb3939&scope.project.id=b9c6b20f7b7a4967ada2d4c9d25dd94a&role.id=af3529d63bae4650be7b8dc67b3ed527 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:01.805366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acc6f8-a6d0-43d6-9cbf-8c7752f2c8cc None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:01.805810 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acc6f8-a6d0-43d6-9cbf-8c7752f2c8cc None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:01.805938 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acc6f8-a6d0-43d6-9cbf-8c7752f2c8cc None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:01.807390 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 87/187] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:01.812552 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9ac64-4a73-404e-ac12-d669840e1b1f None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:01.812914 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9ac64-4a73-404e-ac12-d669840e1b1f None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:01.813282 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9ac64-4a73-404e-ac12-d669840e1b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:01.833976 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-f9d9ac64-4a73-404e-ac12-d669840e1b1f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:01.835910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-f9d9ac64-4a73-404e-ac12-d669840e1b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:01.894281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:01.915961 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 101/188] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:01.951152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beca1b5-04c2-41c9-929e-e65e3f4e2dd0 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:01.951492 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beca1b5-04c2-41c9-929e-e65e3f4e2dd0 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:01.951716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beca1b5-04c2-41c9-929e-e65e3f4e2dd0 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:01.953329 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 88/189] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:01.959191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a5010-54ca-46b7-a492-eb3a351f9a9c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:01.959517 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a5010-54ca-46b7-a492-eb3a351f9a9c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:01.959750 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a5010-54ca-46b7-a492-eb3a351f9a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:01.979121 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-b39a5010-54ca-46b7-a492-eb3a351f9a9c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:01.980340 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b39a5010-54ca-46b7-a492-eb3a351f9a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:02.050676 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 102/190] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:02.054811 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99efe5-99bb-4600-94dc-dbea506419af None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:02.055430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99efe5-99bb-4600-94dc-dbea506419af None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:02.055532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99efe5-99bb-4600-94dc-dbea506419af None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:02.057037 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 89/191] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:02.062041 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba829-4d24-45dc-81e5-09a8dfc7b5c6 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:02.062336 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba829-4d24-45dc-81e5-09a8dfc7b5c6 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:02.062616 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba829-4d24-45dc-81e5-09a8dfc7b5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:02.081787 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-f0fba829-4d24-45dc-81e5-09a8dfc7b5c6 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:02.082990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-f0fba829-4d24-45dc-81e5-09a8dfc7b5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:02.153831 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 103/192] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:35:02.185551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99efe5-99bb-4600-94dc-dbea506419af None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:02.288584 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99efe5-99bb-4600-94dc-dbea506419af None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:02.305556 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:02.322821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445c1db-17fd-4c42-b2f8-9595a0bd9236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:02.323476 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445c1db-17fd-4c42-b2f8-9595a0bd9236 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:02.323752 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445c1db-17fd-4c42-b2f8-9595a0bd9236 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:02.324024 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445c1db-17fd-4c42-b2f8-9595a0bd9236 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:02.334385 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445c1db-17fd-4c42-b2f8-9595a0bd9236 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:02.341403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-2445c1db-17fd-4c42-b2f8-9595a0bd9236 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:35:02.341403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:02.341403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:02.341403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:02.341403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:02.341403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:02.341403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:02.341403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:02.341403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:02.341403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:02.341403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:02.341403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:02.341403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:02.341403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:02.341403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:02.341403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:02.342293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:02.342293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:02.342293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:02.342293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:02.342293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:02.342293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:02.342293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:02.342293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:02.342293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:02.342293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:02.342293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:02.342293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:02.342293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:02.342293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:02.342293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:02.342293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:02.343782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:02.343782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:02.343782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:02.343782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:02.343782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:02.343782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:02.343782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:02.343782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:02.343782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:02.343782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:02.343782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:35:02.343782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:02.343782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 90/193] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:02.348471 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fba829-4d24-45dc-81e5-09a8dfc7b5c6 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:02.385726 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:02.426265 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445c1db-17fd-4c42-b2f8-9595a0bd9236 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:02.458777 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fba829-4d24-45dc-81e5-09a8dfc7b5c6 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:02.494443 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984d7c6-bf00-470e-8da4-4b5c4b7cce98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:02.495064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984d7c6-bf00-470e-8da4-4b5c4b7cce98 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:02.495343 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984d7c6-bf00-470e-8da4-4b5c4b7cce98 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:02.495617 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984d7c6-bf00-470e-8da4-4b5c4b7cce98 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:02.496314 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984d7c6-bf00-470e-8da4-4b5c4b7cce98 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:02.496616 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984d7c6-bf00-470e-8da4-4b5c4b7cce98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:02.498010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984d7c6-bf00-470e-8da4-4b5c4b7cce98 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:02.506283 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 104/194] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:35:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:02.507828 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984d7c6-bf00-470e-8da4-4b5c4b7cce98 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:02.531483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445c1db-17fd-4c42-b2f8-9595a0bd9236 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:02.567085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d0a40-77cc-4b81-b82a-6d7cec538162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:02.567638 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d0a40-77cc-4b81-b82a-6d7cec538162 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:02.567919 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d0a40-77cc-4b81-b82a-6d7cec538162 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:02.568219 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d0a40-77cc-4b81-b82a-6d7cec538162 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:02.578898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d0a40-77cc-4b81-b82a-6d7cec538162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:02.584203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-9d3d0a40-77cc-4b81-b82a-6d7cec538162 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:35:02.584203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:02.584203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:02.584203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:02.584203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:02.584203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:02.584203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:02.584203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:02.584203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:02.584203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:02.584203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:02.584203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:02.584203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:02.584203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:02.584203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:02.584203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:02.585194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:02.585194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:02.585194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:02.585194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:02.585194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:02.585194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:02.585194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:02.585194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:02.585194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:02.585194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:02.585194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:02.585194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:02.585194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:02.585194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:02.585194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:02.585194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:02.586006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:02.586006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:02.586006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:02.586006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:02.586006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:02.586006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:02.586006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:02.586006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:02.586006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:02.586006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:02.586006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:35:02.586006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:02.586006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 91/195] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:35:02.586673 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d0a40-77cc-4b81-b82a-6d7cec538162 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:02.613054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:02.617812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984d7c6-bf00-470e-8da4-4b5c4b7cce98 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:02.630342 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:02.652499 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180a779-301e-4084-9f87-9474ac5059f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:02.653128 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180a779-301e-4084-9f87-9474ac5059f3 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:02.653405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180a779-301e-4084-9f87-9474ac5059f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:02.653673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180a779-301e-4084-9f87-9474ac5059f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:02.654789 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180a779-301e-4084-9f87-9474ac5059f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:02.656283 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180a779-301e-4084-9f87-9474ac5059f3 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:02.657947 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 105/196] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:02.658723 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180a779-301e-4084-9f87-9474ac5059f3 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:02.698196 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d0a40-77cc-4b81-b82a-6d7cec538162 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:02.736589 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58f1cf-2ec7-4050-a465-3d100440b700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:02.737220 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58f1cf-2ec7-4050-a465-3d100440b700 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:02.737501 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58f1cf-2ec7-4050-a465-3d100440b700 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:02.737770 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58f1cf-2ec7-4050-a465-3d100440b700 None admin] PATH_INFO: `/v3/users/55b0e2055070453a9194407289dfe7bd` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:02.751726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58f1cf-2ec7-4050-a465-3d100440b700 None admin] RBAC: Authorizing `identity:get_user(user_id=55b0e2055070453a9194407289dfe7bd)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:02.753243 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58f1cf-2ec7-4050-a465-3d100440b700 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:02.754930 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 92/197] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:35:02 2022] GET /identity/v3/users/55b0e2055070453a9194407289dfe7bd => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:02.755689 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58f1cf-2ec7-4050-a465-3d100440b700 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:02.773308 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180a779-301e-4084-9f87-9474ac5059f3 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:02.809955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81101b7d-d99e-4c1b-a4fc-9893941490ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:02.810563 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81101b7d-d99e-4c1b-a4fc-9893941490ff None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:02.810840 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81101b7d-d99e-4c1b-a4fc-9893941490ff None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:02.811109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81101b7d-d99e-4c1b-a4fc-9893941490ff None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:02.811789 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81101b7d-d99e-4c1b-a4fc-9893941490ff None admin] RBAC: Adding query filter params (name=member) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:02.812112 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81101b7d-d99e-4c1b-a4fc-9893941490ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:02.813498 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81101b7d-d99e-4c1b-a4fc-9893941490ff None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:02.827946 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 106/198] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:35:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:02.827946 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81101b7d-d99e-4c1b-a4fc-9893941490ff None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:02.853436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:02.872418 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:02.878675 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58f1cf-2ec7-4050-a465-3d100440b700 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:02.917089 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375753ac-9af4-4f22-a2cd-30ee17e1d803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:02.917706 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375753ac-9af4-4f22-a2cd-30ee17e1d803 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:02.917983 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375753ac-9af4-4f22-a2cd-30ee17e1d803 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:02.918255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375753ac-9af4-4f22-a2cd-30ee17e1d803 None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:02.918915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375753ac-9af4-4f22-a2cd-30ee17e1d803 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:02.920396 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375753ac-9af4-4f22-a2cd-30ee17e1d803 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:02.938659 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81101b7d-d99e-4c1b-a4fc-9893941490ff None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:02.942455 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 93/199] 10.222.0.36 () {60 vars in 1132 bytes} [Sun Oct 30 00:35:02 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:35:02.945417 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375753ac-9af4-4f22-a2cd-30ee17e1d803 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:02.979645 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967eeed1-42e8-4eaf-909f-21d49d2af63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:02.980334 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967eeed1-42e8-4eaf-909f-21d49d2af63e None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:02.980611 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967eeed1-42e8-4eaf-909f-21d49d2af63e None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:02.980901 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967eeed1-42e8-4eaf-909f-21d49d2af63e None admin] PATH_INFO: `/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:02.982167 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967eeed1-42e8-4eaf-909f-21d49d2af63e None admin] RBAC: Authorizing `identity:get_project(project_id=b9c6b20f7b7a4967ada2d4c9d25dd94a)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:02.983683 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967eeed1-42e8-4eaf-909f-21d49d2af63e None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:02.985651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 107/200] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:35:02 2022] GET /identity/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:02.991392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967eeed1-42e8-4eaf-909f-21d49d2af63e None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:03.072423 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375753ac-9af4-4f22-a2cd-30ee17e1d803 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:03.108579 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:03.110459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7d966-2a8e-4752-ba32-4cb57af5509a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:03.111235 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7d966-2a8e-4752-ba32-4cb57af5509a None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:03.111578 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7d966-2a8e-4752-ba32-4cb57af5509a None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:03.111989 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7d966-2a8e-4752-ba32-4cb57af5509a None admin] PATH_INFO: `/v3/users/f49944cb82fb436cab591a97ba89b2e6` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:03.113252 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7d966-2a8e-4752-ba32-4cb57af5509a None admin] RBAC: Authorizing `identity:get_user(user_id=f49944cb82fb436cab591a97ba89b2e6)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:03.114717 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7d966-2a8e-4752-ba32-4cb57af5509a None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:03.116470 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 94/201] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:35:02 2022] GET /identity/v3/users/f49944cb82fb436cab591a97ba89b2e6 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:03.117829 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967eeed1-42e8-4eaf-909f-21d49d2af63e None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:03.122984 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7d966-2a8e-4752-ba32-4cb57af5509a None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:03.147180 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:03.161215 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bc69b-8d16-45a6-b390-03de72b29841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:03.162160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bc69b-8d16-45a6-b390-03de72b29841 None admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:03.162441 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bc69b-8d16-45a6-b390-03de72b29841 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:03.162711 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bc69b-8d16-45a6-b390-03de72b29841 None admin] PATH_INFO: `/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a/users/55b0e2055070453a9194407289dfe7bd/roles/2e58f1f4fb154236982461a05ee16601` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:03.166082 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569bc69b-8d16-45a6-b390-03de72b29841 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9c6b20f7b7a4967ada2d4c9d25dd94a, user_id=55b0e2055070453a9194407289dfe7bd, role_id=2e58f1f4fb154236982461a05ee16601)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:03.167564 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569bc69b-8d16-45a6-b390-03de72b29841 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:03.188583 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 108/202] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:35:02 2022] PUT /identity/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a/users/55b0e2055070453a9194407289dfe7bd/roles/2e58f1f4fb154236982461a05ee16601 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:35:03.257156 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7d966-2a8e-4752-ba32-4cb57af5509a None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:03.302192 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29deafca-b54e-4b38-8d06-d2dc7062af5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:03.302836 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29deafca-b54e-4b38-8d06-d2dc7062af5b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:03.303115 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29deafca-b54e-4b38-8d06-d2dc7062af5b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:03.303389 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29deafca-b54e-4b38-8d06-d2dc7062af5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:03.390098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29deafca-b54e-4b38-8d06-d2dc7062af5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:03.391520 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29deafca-b54e-4b38-8d06-d2dc7062af5b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:03.415466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:03.508017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 95/203] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:35:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:35:03.513602 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bc69b-8d16-45a6-b390-03de72b29841 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:03.635878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bc69b-8d16-45a6-b390-03de72b29841 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:03.675606 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc90e9-abea-43fd-af2a-b82e6ee4d827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:03.676294 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc90e9-abea-43fd-af2a-b82e6ee4d827 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:03.676581 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc90e9-abea-43fd-af2a-b82e6ee4d827 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:03.676859 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc90e9-abea-43fd-af2a-b82e6ee4d827 None admin] PATH_INFO: `/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:03.678019 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc90e9-abea-43fd-af2a-b82e6ee4d827 None admin] RBAC: Authorizing `identity:get_project(project_id=b9c6b20f7b7a4967ada2d4c9d25dd94a)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:03.680133 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc90e9-abea-43fd-af2a-b82e6ee4d827 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:03.682180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 109/204] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:35:03 2022] GET /identity/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:03.688218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29deafca-b54e-4b38-8d06-d2dc7062af5b None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:03.801838 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29deafca-b54e-4b38-8d06-d2dc7062af5b None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:03.828287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:03.846035 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ac682-22ed-401b-a3ef-e3c4d65fa69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:03.846663 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ac682-22ed-401b-a3ef-e3c4d65fa69b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:03.846944 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ac682-22ed-401b-a3ef-e3c4d65fa69b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:03.847313 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ac682-22ed-401b-a3ef-e3c4d65fa69b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:03.848112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ac682-22ed-401b-a3ef-e3c4d65fa69b None admin] RBAC: Adding query filter params (role.id=2e0593ed85a14197a0eaf37135a01787, scope.project.id=b9c6b20f7b7a4967ada2d4c9d25dd94a, user.id=f49944cb82fb436cab591a97ba89b2e6) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:03.848541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ac682-22ed-401b-a3ef-e3c4d65fa69b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:03.850080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ac682-22ed-401b-a3ef-e3c4d65fa69b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:03.857590 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 96/205] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:35:03 2022] GET /identity/v3/role_assignments?user.id=f49944cb82fb436cab591a97ba89b2e6&scope.project.id=b9c6b20f7b7a4967ada2d4c9d25dd94a&role.id=2e0593ed85a14197a0eaf37135a01787 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:05.347544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f321e4-f42c-4ff7-a925-1d10811b9bd2 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:05.348075 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f321e4-f42c-4ff7-a925-1d10811b9bd2 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:05.348075 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f321e4-f42c-4ff7-a925-1d10811b9bd2 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:05.352126 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 110/206] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:05.355276 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa33b0-15b2-4ba7-a837-65323b23f64f None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:05.355787 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa33b0-15b2-4ba7-a837-65323b23f64f None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:05.355876 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa33b0-15b2-4ba7-a837-65323b23f64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:05.375559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-f4aa33b0-15b2-4ba7-a837-65323b23f64f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:05.376776 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f4aa33b0-15b2-4ba7-a837-65323b23f64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:05.414962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db228e47-347e-4a9d-90bc-551e45ed6ecb None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:05.415255 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db228e47-347e-4a9d-90bc-551e45ed6ecb None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:05.415528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db228e47-347e-4a9d-90bc-551e45ed6ecb None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:05.416922 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 111/207] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:35:05.422145 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165fd16-5ce8-4802-b402-6e3e86152f8c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:05.422433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165fd16-5ce8-4802-b402-6e3e86152f8c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:05.422712 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165fd16-5ce8-4802-b402-6e3e86152f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:05.441592 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-2165fd16-5ce8-4802-b402-6e3e86152f8c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:05.442793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-2165fd16-5ce8-4802-b402-6e3e86152f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:05.452796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 97/208] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:05.461413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92813f40-fda3-40fc-a319-70425207e51b None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:05.461760 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92813f40-fda3-40fc-a319-70425207e51b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:05.461961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92813f40-fda3-40fc-a319-70425207e51b None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:05.464038 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 98/209] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:35:05.469060 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392b9b9-cb5b-4597-9e47-197e632b022c None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:05.469353 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392b9b9-cb5b-4597-9e47-197e632b022c None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:05.469637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392b9b9-cb5b-4597-9e47-197e632b022c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:05.472437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:05.489042 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-0392b9b9-cb5b-4597-9e47-197e632b022c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:05.490246 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-0392b9b9-cb5b-4597-9e47-197e632b022c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:05.532901 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 112/210] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:35:05.563519 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 99/211] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:35:05.749243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2165fd16-5ce8-4802-b402-6e3e86152f8c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:05.815317 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392b9b9-cb5b-4597-9e47-197e632b022c None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:05.863925 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:05.866592 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2165fd16-5ce8-4802-b402-6e3e86152f8c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:05.901628 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31c4c9-4c45-4d05-97af-87c8079f06d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:05.902264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31c4c9-4c45-4d05-97af-87c8079f06d8 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:05.902553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31c4c9-4c45-4d05-97af-87c8079f06d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:05.902824 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31c4c9-4c45-4d05-97af-87c8079f06d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:05.913343 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31c4c9-4c45-4d05-97af-87c8079f06d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:05.918897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-1f31c4c9-4c45-4d05-97af-87c8079f06d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:35:05.918897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:05.918897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:05.918897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:05.918897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:05.918897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:05.918897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:05.918897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:05.918897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:05.918897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:05.918897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:05.918897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:05.918897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:05.918897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:05.918897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:05.918897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:05.920042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:05.920042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:05.920042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:05.920042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:05.920042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:05.920042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:05.920042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:05.920042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:05.920042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:05.920042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:05.920042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:05.920042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:05.920042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:05.920042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:05.920042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:05.920042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:05.921269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:05.921269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:05.921269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:05.921269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:05.921269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:05.921269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:05.921269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:05.921269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:05.921269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:05.921269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:05.921269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:35:05.921269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:05.921269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 113/212] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:05.921269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31c4c9-4c45-4d05-97af-87c8079f06d8 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:05.928523 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392b9b9-cb5b-4597-9e47-197e632b022c None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:05.938257 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:05.963134 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a8fab-65f9-48ab-b8f9-832212247abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:05.963774 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a8fab-65f9-48ab-b8f9-832212247abc None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:05.964045 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a8fab-65f9-48ab-b8f9-832212247abc None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:05.964344 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6a8fab-65f9-48ab-b8f9-832212247abc None admin] PATH_INFO: `/v3/domains/default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:05.965452 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a8fab-65f9-48ab-b8f9-832212247abc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:05.966928 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6a8fab-65f9-48ab-b8f9-832212247abc None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:05.968534 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 100/213] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:05.969296 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a8fab-65f9-48ab-b8f9-832212247abc None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:06.025016 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31c4c9-4c45-4d05-97af-87c8079f06d8 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:06.059307 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5504f1-d1a3-421e-9ee7-fc80091c6987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:06.059965 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5504f1-d1a3-421e-9ee7-fc80091c6987 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:06.060270 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5504f1-d1a3-421e-9ee7-fc80091c6987 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:06.060548 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5504f1-d1a3-421e-9ee7-fc80091c6987 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:06.069200 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6a8fab-65f9-48ab-b8f9-832212247abc None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:06.070873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5504f1-d1a3-421e-9ee7-fc80091c6987 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:06.075741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-ef5504f1-d1a3-421e-9ee7-fc80091c6987 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:35:06.075741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:06.075741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:06.075741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:06.075741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:06.075741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:06.075741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:06.075741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:06.075741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:06.075741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:06.075741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:06.075741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:06.075741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:06.075741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:06.075741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:06.075741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:06.076732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:06.076732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:06.076732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:06.076732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:06.076732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:06.076732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:06.076732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:06.076732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:06.076732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:06.076732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:06.076732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:06.076732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:06.076732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:06.076732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:06.076732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:06.076732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:06.077758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:06.077758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:06.077758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:06.077758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:06.077758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:06.077758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:06.077758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:06.077758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:06.077758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:06.077758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:06.077758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:35:06.077758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:06.077758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 114/214] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:06.078691 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5504f1-d1a3-421e-9ee7-fc80091c6987 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:06.085259 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:06.107881 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c316507-0917-4f9e-adcd-607024d35415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:06.108303 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c316507-0917-4f9e-adcd-607024d35415 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:06.108587 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c316507-0917-4f9e-adcd-607024d35415 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:06.108858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c316507-0917-4f9e-adcd-607024d35415 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:06.109522 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c316507-0917-4f9e-adcd-607024d35415 None admin] RBAC: Adding query filter params (name=member) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:06.109824 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c316507-0917-4f9e-adcd-607024d35415 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:06.111203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c316507-0917-4f9e-adcd-607024d35415 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:06.119395 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 101/215] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:35:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:06.120032 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c316507-0917-4f9e-adcd-607024d35415 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:06.158987 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:06.185014 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5504f1-d1a3-421e-9ee7-fc80091c6987 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:06.224522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e3dc2-4983-40b8-b89d-869097ec786c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:06.225137 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e3dc2-4983-40b8-b89d-869097ec786c None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:06.225458 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e3dc2-4983-40b8-b89d-869097ec786c None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:06.225665 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c316507-0917-4f9e-adcd-607024d35415 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:06.225766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e3dc2-4983-40b8-b89d-869097ec786c None admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:06.226708 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e3dc2-4983-40b8-b89d-869097ec786c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:06.228167 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e3dc2-4983-40b8-b89d-869097ec786c None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:06.250474 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 115/216] 10.222.0.36 () {60 vars in 1132 bytes} [Sun Oct 30 00:35:06 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:35:06.257830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e3dc2-4983-40b8-b89d-869097ec786c None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:06.263225 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747180e-2a23-47d2-bc2c-26b3712744da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:06.263923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747180e-2a23-47d2-bc2c-26b3712744da None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:06.264269 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747180e-2a23-47d2-bc2c-26b3712744da None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:06.264547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747180e-2a23-47d2-bc2c-26b3712744da None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:06.265229 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747180e-2a23-47d2-bc2c-26b3712744da None admin] RBAC: Adding query filter params (name=reader) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:06.265538 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747180e-2a23-47d2-bc2c-26b3712744da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:06.267020 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747180e-2a23-47d2-bc2c-26b3712744da None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:06.277841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 102/217] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:35:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:06.283858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747180e-2a23-47d2-bc2c-26b3712744da None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:06.326857 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:06.385686 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e3dc2-4983-40b8-b89d-869097ec786c None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:06.412147 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747180e-2a23-47d2-bc2c-26b3712744da None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:06.421438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:06.428859 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403303d-961a-473c-8eca-784ec491e0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:06.429888 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403303d-961a-473c-8eca-784ec491e0cb None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:06.430254 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403303d-961a-473c-8eca-784ec491e0cb None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:06.430751 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403303d-961a-473c-8eca-784ec491e0cb None admin] PATH_INFO: `/v3/users/86cafae7d8704641b02c70022f38982a` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:06.448251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3403303d-961a-473c-8eca-784ec491e0cb None admin] RBAC: Authorizing `identity:get_user(user_id=86cafae7d8704641b02c70022f38982a)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:06.450096 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3403303d-961a-473c-8eca-784ec491e0cb None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:06.451862 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 116/218] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:35:06 2022] GET /identity/v3/users/86cafae7d8704641b02c70022f38982a => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:06.457658 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403303d-961a-473c-8eca-784ec491e0cb None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:06.462923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8210e-d4a0-49af-994a-7db913b01025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:06.463577 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8210e-d4a0-49af-994a-7db913b01025 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:06.463856 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8210e-d4a0-49af-994a-7db913b01025 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:06.464180 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8210e-d4a0-49af-994a-7db913b01025 None admin] PATH_INFO: `/v3/users/55b0e2055070453a9194407289dfe7bd` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:06.465330 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8210e-d4a0-49af-994a-7db913b01025 None admin] RBAC: Authorizing `identity:get_user(user_id=55b0e2055070453a9194407289dfe7bd)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:06.466778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8210e-d4a0-49af-994a-7db913b01025 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:06.468463 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 103/219] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:35:06 2022] GET /identity/v3/users/55b0e2055070453a9194407289dfe7bd => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:06.474414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8210e-d4a0-49af-994a-7db913b01025 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:06.589715 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403303d-961a-473c-8eca-784ec491e0cb None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:06.614632 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8210e-d4a0-49af-994a-7db913b01025 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:06.619808 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:06.638005 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6dd97f-fe63-43c6-83d4-16402151cc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:06.638634 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6dd97f-fe63-43c6-83d4-16402151cc8c None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:06.638915 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6dd97f-fe63-43c6-83d4-16402151cc8c None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:06.639194 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6dd97f-fe63-43c6-83d4-16402151cc8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:06.639938 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6dd97f-fe63-43c6-83d4-16402151cc8c None admin] RBAC: Adding query filter params (role.id=2e0593ed85a14197a0eaf37135a01787, user.id=86cafae7d8704641b02c70022f38982a, scope.system=all) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:06.640283 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6dd97f-fe63-43c6-83d4-16402151cc8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:06.641765 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6dd97f-fe63-43c6-83d4-16402151cc8c None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:06.646995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 117/220] 10.222.0.36 () {56 vars in 1308 bytes} [Sun Oct 30 00:35:06 2022] GET /identity/v3/role_assignments?user.id=86cafae7d8704641b02c70022f38982a&scope.system=all&role.id=2e0593ed85a14197a0eaf37135a01787 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:06.659171 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb956e7-42f3-41e2-92b5-b7ac1ee7119b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:06.659809 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb956e7-42f3-41e2-92b5-b7ac1ee7119b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:06.660115 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb956e7-42f3-41e2-92b5-b7ac1ee7119b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:06.660395 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb956e7-42f3-41e2-92b5-b7ac1ee7119b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:06.743937 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb956e7-42f3-41e2-92b5-b7ac1ee7119b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:06.745490 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb956e7-42f3-41e2-92b5-b7ac1ee7119b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:06.870014 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 104/221] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:35:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 30 00:35:06.876161 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6dd97f-fe63-43c6-83d4-16402151cc8c None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:06.935663 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:06.992155 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6dd97f-fe63-43c6-83d4-16402151cc8c None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:07.030428 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce2d51-9d10-4a94-b1cb-fa1eaa1cdc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:07.031442 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce2d51-9d10-4a94-b1cb-fa1eaa1cdc6b None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:07.031939 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce2d51-9d10-4a94-b1cb-fa1eaa1cdc6b None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:07.032475 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce2d51-9d10-4a94-b1cb-fa1eaa1cdc6b None admin] PATH_INFO: `/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:07.033826 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce2d51-9d10-4a94-b1cb-fa1eaa1cdc6b None admin] RBAC: Authorizing `identity:get_project(project_id=b9c6b20f7b7a4967ada2d4c9d25dd94a)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:07.035541 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce2d51-9d10-4a94-b1cb-fa1eaa1cdc6b None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:07.038136 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 118/222] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:35:06 2022] GET /identity/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:07.048151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb956e7-42f3-41e2-92b5-b7ac1ee7119b None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:07.052103 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:07.178392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb956e7-42f3-41e2-92b5-b7ac1ee7119b None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:07.217788 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3cae6-c271-41b1-9ec6-d67d8b5550f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:07.218437 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3cae6-c271-41b1-9ec6-d67d8b5550f3 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:07.218692 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3cae6-c271-41b1-9ec6-d67d8b5550f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:07.218965 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3cae6-c271-41b1-9ec6-d67d8b5550f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:07.219704 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3cae6-c271-41b1-9ec6-d67d8b5550f3 None admin] RBAC: Adding query filter params (role.id=2e58f1f4fb154236982461a05ee16601, scope.project.id=b9c6b20f7b7a4967ada2d4c9d25dd94a, user.id=55b0e2055070453a9194407289dfe7bd) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:07.220011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3cae6-c271-41b1-9ec6-d67d8b5550f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:07.221483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3cae6-c271-41b1-9ec6-d67d8b5550f3 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:07.228910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 105/223] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:35:07 2022] GET /identity/v3/role_assignments?user.id=55b0e2055070453a9194407289dfe7bd&scope.project.id=b9c6b20f7b7a4967ada2d4c9d25dd94a&role.id=2e58f1f4fb154236982461a05ee16601 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:08.697803 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782a4e7-a49f-4269-b28a-92bf1d94135b None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:08.698355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782a4e7-a49f-4269-b28a-92bf1d94135b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:08.698355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782a4e7-a49f-4269-b28a-92bf1d94135b None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:08.699853 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 119/224] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:08.705065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391853ec-60c0-4914-b9b3-5c9c5f8941d9 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:08.705462 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391853ec-60c0-4914-b9b3-5c9c5f8941d9 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:08.705638 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391853ec-60c0-4914-b9b3-5c9c5f8941d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:08.725587 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-391853ec-60c0-4914-b9b3-5c9c5f8941d9 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:08.727513 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b21e51-52ce-4f81-bb2d-07dd14740ea6 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:08.727640 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-391853ec-60c0-4914-b9b3-5c9c5f8941d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:08.727807 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b21e51-52ce-4f81-bb2d-07dd14740ea6 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:08.728070 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b21e51-52ce-4f81-bb2d-07dd14740ea6 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:08.729704 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 120/225] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:35:08.735213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a81501-ea12-46f7-80e4-cb2b2e0529a8 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:08.735494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a81501-ea12-46f7-80e4-cb2b2e0529a8 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:08.735772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a81501-ea12-46f7-80e4-cb2b2e0529a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:08.755655 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-a8a81501-ea12-46f7-80e4-cb2b2e0529a8 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:08.756898 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a8a81501-ea12-46f7-80e4-cb2b2e0529a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:08.799364 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:08.816401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 106/226] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:08.830971 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 121/227] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:35:09.010466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c083151-fd58-49af-abeb-83f5f4a5e211 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:09.010791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c083151-fd58-49af-abeb-83f5f4a5e211 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:09.010984 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c083151-fd58-49af-abeb-83f5f4a5e211 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:09.012455 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 107/228] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:09.017504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4331fe-d730-4c15-8cd1-26efa3811e6a None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:09.017792 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4331fe-d730-4c15-8cd1-26efa3811e6a None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:09.018063 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4331fe-d730-4c15-8cd1-26efa3811e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:09.037316 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-0e4331fe-d730-4c15-8cd1-26efa3811e6a None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:09.038483 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-0e4331fe-d730-4c15-8cd1-26efa3811e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:09.085210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:09.087297 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c083151-fd58-49af-abeb-83f5f4a5e211 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:09.109086 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 122/229] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:09.109920 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4331fe-d730-4c15-8cd1-26efa3811e6a None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:09.203431 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c083151-fd58-49af-abeb-83f5f4a5e211 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:09.214753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4331fe-d730-4c15-8cd1-26efa3811e6a None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:09.241152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897935d-ea21-4506-96aa-bbccbfa912ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:09.242514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897935d-ea21-4506-96aa-bbccbfa912ea None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:09.242762 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897935d-ea21-4506-96aa-bbccbfa912ea None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:09.243181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897935d-ea21-4506-96aa-bbccbfa912ea None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:09.250959 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c84b19-90c6-421a-b4c0-d13fbe16ee8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:09.251618 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c84b19-90c6-421a-b4c0-d13fbe16ee8f None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:09.251902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c84b19-90c6-421a-b4c0-d13fbe16ee8f None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:09.252213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c84b19-90c6-421a-b4c0-d13fbe16ee8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:09.253336 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c84b19-90c6-421a-b4c0-d13fbe16ee8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:09.254790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c84b19-90c6-421a-b4c0-d13fbe16ee8f None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:09.254999 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c897935d-ea21-4506-96aa-bbccbfa912ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:09.256589 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 123/230] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:09.260279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-c897935d-ea21-4506-96aa-bbccbfa912ea None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:35:09.260279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:09.260279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:09.260279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:09.260279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:09.260279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:09.260279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:09.260279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:09.260279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:09.260279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:09.260279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:09.260279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:09.260279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:09.260279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:09.260279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:09.260279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:09.261226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:09.261226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:09.261226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:09.261226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:09.261226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:09.261226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:09.261226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:09.261226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:09.261226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:09.261226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:09.261226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:09.261226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:09.261226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:09.261226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:09.261226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:09.261226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:09.262053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:09.262053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:09.262053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:09.262053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:09.262053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:09.262053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:09.262053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:09.262053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:09.262053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:09.262053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:09.262053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:35:09.262053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:09.262053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 108/231] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:09.262707 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c84b19-90c6-421a-b4c0-d13fbe16ee8f None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:09.266933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897935d-ea21-4506-96aa-bbccbfa912ea None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:09.322538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:09.333304 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:09.390339 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897935d-ea21-4506-96aa-bbccbfa912ea None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:09.393219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c84b19-90c6-421a-b4c0-d13fbe16ee8f None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:09.427682 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645f6af-fafa-4ee1-a493-d758d0cc2fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:09.428343 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645f6af-fafa-4ee1-a493-d758d0cc2fe9 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:09.428625 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645f6af-fafa-4ee1-a493-d758d0cc2fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:09.428903 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645f6af-fafa-4ee1-a493-d758d0cc2fe9 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:09.429576 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645f6af-fafa-4ee1-a493-d758d0cc2fe9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:09.429877 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645f6af-fafa-4ee1-a493-d758d0cc2fe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:09.431285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645f6af-fafa-4ee1-a493-d758d0cc2fe9 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:09.431407 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2bb126-8c8e-4b48-a24f-568c09dc3638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:09.431988 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2bb126-8c8e-4b48-a24f-568c09dc3638 None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:09.432294 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2bb126-8c8e-4b48-a24f-568c09dc3638 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:09.432568 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2bb126-8c8e-4b48-a24f-568c09dc3638 None admin] PATH_INFO: `/v3/groups` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:09.433204 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2bb126-8c8e-4b48-a24f-568c09dc3638 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:09.434647 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2bb126-8c8e-4b48-a24f-568c09dc3638 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:09.440637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 109/232] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:35:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:09.441497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645f6af-fafa-4ee1-a493-d758d0cc2fe9 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:09.454740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 124/233] 10.222.0.36 () {60 vars in 1134 bytes} [Sun Oct 30 00:35:09 2022] POST /identity/v3/groups => generated 229 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:35:09.455455 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2bb126-8c8e-4b48-a24f-568c09dc3638 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:09.583881 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645f6af-fafa-4ee1-a493-d758d0cc2fe9 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:09.601612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2bb126-8c8e-4b48-a24f-568c09dc3638 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:09.615830 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:09.621370 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:09.628739 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5132fb3-a95d-4287-befa-9b2d82cac1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:09.629400 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5132fb3-a95d-4287-befa-9b2d82cac1c5 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:09.629679 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5132fb3-a95d-4287-befa-9b2d82cac1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:09.629954 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5132fb3-a95d-4287-befa-9b2d82cac1c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:09.643105 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5132fb3-a95d-4287-befa-9b2d82cac1c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:09.648931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-f5132fb3-a95d-4287-befa-9b2d82cac1c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:35:09.648931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:09.648931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:09.648931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:09.648931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:09.648931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:09.648931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:09.648931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:09.648931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:09.648931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:09.648931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:09.648931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:09.648931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:09.648931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:09.648931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:09.648931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:09.649910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:09.649910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:09.649910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:09.649910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:09.649910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:09.649910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:09.649910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:09.649910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:09.649910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:09.649910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:09.649910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:09.649910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:09.649910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:09.649910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:09.649910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:09.649910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:09.650707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:09.650707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:09.650707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:09.650707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:09.650707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:09.650707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:09.650707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:09.650707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:09.650707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:09.650707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:09.650707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:35:09.650707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:09.650707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 110/234] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:09.655936 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5132fb3-a95d-4287-befa-9b2d82cac1c5 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:09.668392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e916e31-378f-426c-a5b8-ff96e574cd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:09.669474 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e916e31-378f-426c-a5b8-ff96e574cd22 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:09.669929 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e916e31-378f-426c-a5b8-ff96e574cd22 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:09.670383 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e916e31-378f-426c-a5b8-ff96e574cd22 None admin] PATH_INFO: `/v3/users/d539a888f6084a2d9afa689aed43bbff` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:09.693067 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e916e31-378f-426c-a5b8-ff96e574cd22 None admin] RBAC: Authorizing `identity:get_user(user_id=d539a888f6084a2d9afa689aed43bbff)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:09.697337 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e916e31-378f-426c-a5b8-ff96e574cd22 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:09.700244 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 125/235] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:35:09 2022] GET /identity/v3/users/d539a888f6084a2d9afa689aed43bbff => generated 293 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:09.713073 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e916e31-378f-426c-a5b8-ff96e574cd22 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:09.780045 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5132fb3-a95d-4287-befa-9b2d82cac1c5 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:09.819197 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c944e69-2780-403a-9c91-36520f1eaadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:09.819825 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c944e69-2780-403a-9c91-36520f1eaadb None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:09.820689 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c944e69-2780-403a-9c91-36520f1eaadb None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:09.821586 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c944e69-2780-403a-9c91-36520f1eaadb None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:09.821586 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c944e69-2780-403a-9c91-36520f1eaadb None admin] RBAC: Adding query filter params (name=member) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:09.822470 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c944e69-2780-403a-9c91-36520f1eaadb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:09.823265 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c944e69-2780-403a-9c91-36520f1eaadb None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:09.839491 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 111/236] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:35:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:09.840823 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c944e69-2780-403a-9c91-36520f1eaadb None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:09.861112 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e916e31-378f-426c-a5b8-ff96e574cd22 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:09.895076 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:09.918785 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d1a23-d0ea-4bbc-8f29-b59404e2a460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:09.919741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d1a23-d0ea-4bbc-8f29-b59404e2a460 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:09.920235 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d1a23-d0ea-4bbc-8f29-b59404e2a460 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:09.920669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d1a23-d0ea-4bbc-8f29-b59404e2a460 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:09.921831 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d1a23-d0ea-4bbc-8f29-b59404e2a460 None admin] RBAC: Adding query filter params (role.id=2e58f1f4fb154236982461a05ee16601, user.id=d539a888f6084a2d9afa689aed43bbff, scope.system=all) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:09.922321 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d1a23-d0ea-4bbc-8f29-b59404e2a460 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:09.924788 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d1a23-d0ea-4bbc-8f29-b59404e2a460 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:09.940139 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 126/237] 10.222.0.36 () {56 vars in 1308 bytes} [Sun Oct 30 00:35:09 2022] GET /identity/v3/role_assignments?user.id=d539a888f6084a2d9afa689aed43bbff&scope.system=all&role.id=2e58f1f4fb154236982461a05ee16601 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:09.973593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c944e69-2780-403a-9c91-36520f1eaadb None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:10.026617 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b198c-3018-4370-b043-78ff8806baf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:10.027280 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b198c-3018-4370-b043-78ff8806baf5 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:10.027557 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b198c-3018-4370-b043-78ff8806baf5 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:10.027827 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b198c-3018-4370-b043-78ff8806baf5 None admin] PATH_INFO: `/v3/users/86cafae7d8704641b02c70022f38982a` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:10.029086 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b198c-3018-4370-b043-78ff8806baf5 None admin] RBAC: Authorizing `identity:get_user(user_id=86cafae7d8704641b02c70022f38982a)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:10.030597 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b198c-3018-4370-b043-78ff8806baf5 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:10.037439 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 112/238] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:35:09 2022] GET /identity/v3/users/86cafae7d8704641b02c70022f38982a => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:10.039385 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d1a23-d0ea-4bbc-8f29-b59404e2a460 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:10.077033 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:10.158018 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d1a23-d0ea-4bbc-8f29-b59404e2a460 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:10.194068 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c96768-983a-4e2f-b927-eabaf9cd36fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:10.194738 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c96768-983a-4e2f-b927-eabaf9cd36fd None admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:10.195601 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c96768-983a-4e2f-b927-eabaf9cd36fd None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:10.195894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c96768-983a-4e2f-b927-eabaf9cd36fd None admin] PATH_INFO: `/v3/system/users/86cafae7d8704641b02c70022f38982a/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:10.197610 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c96768-983a-4e2f-b927-eabaf9cd36fd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=86cafae7d8704641b02c70022f38982a, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:10.199036 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c96768-983a-4e2f-b927-eabaf9cd36fd None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:10.208730 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 127/239] 10.222.0.36 () {58 vars in 1334 bytes} [Sun Oct 30 00:35:10 2022] PUT /identity/v3/system/users/86cafae7d8704641b02c70022f38982a/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:35:11.743088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3871ca-e087-4bb9-93dd-2c23abb9e4b7 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:11.743701 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3871ca-e087-4bb9-93dd-2c23abb9e4b7 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:11.743701 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3871ca-e087-4bb9-93dd-2c23abb9e4b7 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:11.745156 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 113/240] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:11.750454 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62147b-9390-4ac3-b749-fa7b76fedbc2 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:11.750893 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62147b-9390-4ac3-b749-fa7b76fedbc2 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:11.750989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62147b-9390-4ac3-b749-fa7b76fedbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:11.769551 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-6e62147b-9390-4ac3-b749-fa7b76fedbc2 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:11.770758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-6e62147b-9390-4ac3-b749-fa7b76fedbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:11.842007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 128/241] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:12.109386 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3871ca-e087-4bb9-93dd-2c23abb9e4b7 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:12.228691 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e3cd8-96ad-467a-acd0-573626dd17df None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:12.228993 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e3cd8-96ad-467a-acd0-573626dd17df None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:12.229285 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e3cd8-96ad-467a-acd0-573626dd17df None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:12.230759 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 129/242] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:12.235848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9614c3-c6c5-42ee-939c-57bf128205da None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:12.236162 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9614c3-c6c5-42ee-939c-57bf128205da None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:12.236438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9614c3-c6c5-42ee-939c-57bf128205da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:12.239574 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3871ca-e087-4bb9-93dd-2c23abb9e4b7 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:12.251705 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:12.254449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-3e9614c3-c6c5-42ee-939c-57bf128205da None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:12.255653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-3e9614c3-c6c5-42ee-939c-57bf128205da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:12.280176 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:12.290187 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b88f47-296b-4b75-a6dc-0cd6e735d479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:12.291005 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b88f47-296b-4b75-a6dc-0cd6e735d479 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:12.291114 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b88f47-296b-4b75-a6dc-0cd6e735d479 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:12.291446 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b88f47-296b-4b75-a6dc-0cd6e735d479 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:12.292597 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b88f47-296b-4b75-a6dc-0cd6e735d479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:12.294053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b88f47-296b-4b75-a6dc-0cd6e735d479 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:12.296125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 114/243] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:12.301266 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b88f47-296b-4b75-a6dc-0cd6e735d479 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:12.343079 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 130/244] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:35:12.425215 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b88f47-296b-4b75-a6dc-0cd6e735d479 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:12.434326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a8ed0-8672-4dde-aa13-a42b110e5e7a None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:12.434633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a8ed0-8672-4dde-aa13-a42b110e5e7a None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:12.434909 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a8ed0-8672-4dde-aa13-a42b110e5e7a None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:12.436448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 131/245] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:35:12.442023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e452fc-a347-4b00-99a5-0a8d6fecb1ce None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:12.442311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e452fc-a347-4b00-99a5-0a8d6fecb1ce None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:12.442588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e452fc-a347-4b00-99a5-0a8d6fecb1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:12.462030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-a9e452fc-a347-4b00-99a5-0a8d6fecb1ce None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:12.463237 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a9e452fc-a347-4b00-99a5-0a8d6fecb1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:12.463372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d5608-e37c-4abf-8de1-d0926d74a3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:12.463917 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d5608-e37c-4abf-8de1-d0926d74a3a8 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:12.464236 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d5608-e37c-4abf-8de1-d0926d74a3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:12.464513 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d5608-e37c-4abf-8de1-d0926d74a3a8 None admin] PATH_INFO: `/v3/groups` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:12.465146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d5608-e37c-4abf-8de1-d0926d74a3a8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:12.466595 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d5608-e37c-4abf-8de1-d0926d74a3a8 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:12.486292 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 115/246] 10.222.0.36 () {60 vars in 1134 bytes} [Sun Oct 30 00:35:12 2022] POST /identity/v3/groups => generated 226 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:35:12.544009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 132/247] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:35:12.628243 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d5608-e37c-4abf-8de1-d0926d74a3a8 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:12.675003 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:12.740638 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d5608-e37c-4abf-8de1-d0926d74a3a8 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:12.780170 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453afc2-2c83-460b-b1be-49775f8563c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:12.780812 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453afc2-2c83-460b-b1be-49775f8563c3 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:12.781077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453afc2-2c83-460b-b1be-49775f8563c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:12.781345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453afc2-2c83-460b-b1be-49775f8563c3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:12.794050 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453afc2-2c83-460b-b1be-49775f8563c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:12.798861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-7453afc2-2c83-460b-b1be-49775f8563c3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:35:12.798861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:12.798861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:12.798861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:12.798861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:12.798861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:12.798861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:12.798861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:12.798861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:12.798861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:12.798861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:12.798861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:12.798861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:12.798861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:12.798861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:12.798861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:12.799751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:12.799751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:12.799751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:12.799751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:12.799751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:12.799751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:12.799751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:12.799751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:12.799751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:12.799751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:12.799751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:12.799751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:12.799751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:12.799751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:12.799751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:12.799751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:12.800866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:12.800866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:12.800866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:12.800866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:12.800866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:12.800866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:12.800866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:12.800866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:12.800866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:12.800866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:12.800866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:35:12.800866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:12.800866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 116/248] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:12.806598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e452fc-a347-4b00-99a5-0a8d6fecb1ce None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:12.818781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453afc2-2c83-460b-b1be-49775f8563c3 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:12.939990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:12.959210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:12.961819 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e452fc-a347-4b00-99a5-0a8d6fecb1ce None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:12.968699 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453afc2-2c83-460b-b1be-49775f8563c3 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:13.015862 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76beea3-c647-4300-94ef-04c4b6aadfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:13.016437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76beea3-c647-4300-94ef-04c4b6aadfbb None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:13.016719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76beea3-c647-4300-94ef-04c4b6aadfbb None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:13.016991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76beea3-c647-4300-94ef-04c4b6aadfbb None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:13.019979 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76beea3-c647-4300-94ef-04c4b6aadfbb None admin] RBAC: Adding query filter params (name=reader) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:13.019979 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76beea3-c647-4300-94ef-04c4b6aadfbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:13.020219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76beea3-c647-4300-94ef-04c4b6aadfbb None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:13.028726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656ff36-90f7-4a71-a13b-7b63a2d2ec33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:13.029390 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656ff36-90f7-4a71-a13b-7b63a2d2ec33 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:13.029667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656ff36-90f7-4a71-a13b-7b63a2d2ec33 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:13.029939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656ff36-90f7-4a71-a13b-7b63a2d2ec33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:13.035959 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 133/249] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:35:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:13.043941 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76beea3-c647-4300-94ef-04c4b6aadfbb None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:13.047561 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656ff36-90f7-4a71-a13b-7b63a2d2ec33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:13.052559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-b656ff36-90f7-4a71-a13b-7b63a2d2ec33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:35:13.052559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:13.052559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:13.052559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:13.052559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:13.052559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:13.052559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:13.052559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:13.052559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:13.052559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:13.052559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:13.052559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:13.052559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:13.052559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:13.052559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:13.052559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:13.053471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:13.053471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:13.053471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:13.053471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:13.053471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:13.053471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:13.053471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:13.053471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:13.053471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:13.053471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:13.053471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:13.053471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:13.053471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:13.053471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:13.053471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:13.053471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:13.054287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:13.054287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:13.054287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:13.054287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:13.054287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:13.054287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:13.054287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:13.054287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:13.054287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:13.054287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:13.054287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:35:13.054287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:13.054287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 117/250] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:35:13.060540 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656ff36-90f7-4a71-a13b-7b63a2d2ec33 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:13.194565 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76beea3-c647-4300-94ef-04c4b6aadfbb None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:13.205549 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656ff36-90f7-4a71-a13b-7b63a2d2ec33 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:13.248334 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164c1f5-72ec-4a07-b12c-84d17fd97e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:13.249295 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164c1f5-72ec-4a07-b12c-84d17fd97e17 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:13.249747 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164c1f5-72ec-4a07-b12c-84d17fd97e17 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:13.250190 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164c1f5-72ec-4a07-b12c-84d17fd97e17 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:13.251177 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164c1f5-72ec-4a07-b12c-84d17fd97e17 None admin] RBAC: Adding query filter params (name=member) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:13.251700 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164c1f5-72ec-4a07-b12c-84d17fd97e17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:13.253441 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164c1f5-72ec-4a07-b12c-84d17fd97e17 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:13.258173 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb49dbb-e7aa-4df6-8442-a54248524705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:13.260599 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb49dbb-e7aa-4df6-8442-a54248524705 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:13.260599 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb49dbb-e7aa-4df6-8442-a54248524705 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:13.260599 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb49dbb-e7aa-4df6-8442-a54248524705 None admin] PATH_INFO: `/v3/users/d539a888f6084a2d9afa689aed43bbff` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:13.261295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb49dbb-e7aa-4df6-8442-a54248524705 None admin] RBAC: Authorizing `identity:get_user(user_id=d539a888f6084a2d9afa689aed43bbff)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:13.265122 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb49dbb-e7aa-4df6-8442-a54248524705 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:13.268196 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 134/251] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:35:13 2022] GET /identity/v3/users/d539a888f6084a2d9afa689aed43bbff => generated 293 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:13.270364 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 118/252] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:35:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:13.276677 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb49dbb-e7aa-4df6-8442-a54248524705 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:13.278231 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164c1f5-72ec-4a07-b12c-84d17fd97e17 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:13.298567 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:13.312377 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:13.433537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164c1f5-72ec-4a07-b12c-84d17fd97e17 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:13.443824 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb49dbb-e7aa-4df6-8442-a54248524705 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:13.483794 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827b275-131d-4d31-a80e-850d4cd538a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:13.484485 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b275-131d-4d31-a80e-850d4cd538a0 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:13.484767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b275-131d-4d31-a80e-850d4cd538a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:13.485040 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b275-131d-4d31-a80e-850d4cd538a0 None admin] PATH_INFO: `/v3/users/86cafae7d8704641b02c70022f38982a` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:13.486218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827b275-131d-4d31-a80e-850d4cd538a0 None admin] RBAC: Authorizing `identity:get_user(user_id=86cafae7d8704641b02c70022f38982a)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:13.488379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827b275-131d-4d31-a80e-850d4cd538a0 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:13.489784 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 119/253] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:35:13 2022] GET /identity/v3/users/86cafae7d8704641b02c70022f38982a => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:13.498511 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1197bdd-a1fc-471b-8304-a4fec49e81c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:13.499638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1197bdd-a1fc-471b-8304-a4fec49e81c8 None admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:13.500132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1197bdd-a1fc-471b-8304-a4fec49e81c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:13.500659 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1197bdd-a1fc-471b-8304-a4fec49e81c8 None admin] PATH_INFO: `/v3/system/users/d539a888f6084a2d9afa689aed43bbff/roles/2e58f1f4fb154236982461a05ee16601` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:13.503730 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1197bdd-a1fc-471b-8304-a4fec49e81c8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d539a888f6084a2d9afa689aed43bbff, role_id=2e58f1f4fb154236982461a05ee16601)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:13.504127 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827b275-131d-4d31-a80e-850d4cd538a0 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:13.505583 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1197bdd-a1fc-471b-8304-a4fec49e81c8 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:13.519176 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 135/254] 10.222.0.36 () {58 vars in 1334 bytes} [Sun Oct 30 00:35:13 2022] PUT /identity/v3/system/users/d539a888f6084a2d9afa689aed43bbff/roles/2e58f1f4fb154236982461a05ee16601 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:35:13.617558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:13.634650 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827b275-131d-4d31-a80e-850d4cd538a0 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:13.691863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75560616-2e81-4e0e-88b5-93afbecc9486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:13.693147 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75560616-2e81-4e0e-88b5-93afbecc9486 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:13.693584 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75560616-2e81-4e0e-88b5-93afbecc9486 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:13.694029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75560616-2e81-4e0e-88b5-93afbecc9486 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:13.694992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75560616-2e81-4e0e-88b5-93afbecc9486 None admin] RBAC: Adding query filter params (role.id=2e0593ed85a14197a0eaf37135a01787, user.id=86cafae7d8704641b02c70022f38982a, scope.system=all) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:13.696048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75560616-2e81-4e0e-88b5-93afbecc9486 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:13.698047 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75560616-2e81-4e0e-88b5-93afbecc9486 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:13.720207 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 120/255] 10.222.0.36 () {56 vars in 1308 bytes} [Sun Oct 30 00:35:13 2022] GET /identity/v3/role_assignments?user.id=86cafae7d8704641b02c70022f38982a&scope.system=all&role.id=2e0593ed85a14197a0eaf37135a01787 => generated 510 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:15.209387 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73c881-f00b-4cd2-a883-01abaa9786d6 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:15.210279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73c881-f00b-4cd2-a883-01abaa9786d6 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:15.210699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73c881-f00b-4cd2-a883-01abaa9786d6 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:15.212435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 136/256] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:15.219320 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936295ae-474c-46a8-8eb1-486601195301 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:15.220609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936295ae-474c-46a8-8eb1-486601195301 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:15.221304 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936295ae-474c-46a8-8eb1-486601195301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:15.263641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-936295ae-474c-46a8-8eb1-486601195301 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:15.265983 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-936295ae-474c-46a8-8eb1-486601195301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:15.391514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 121/257] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 30 00:35:15.898418 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73c881-f00b-4cd2-a883-01abaa9786d6 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:15.916754 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73c881-f00b-4cd2-a883-01abaa9786d6 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:15.921533 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb55787-05b3-45a6-899a-f13a1e511085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:15.922195 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb55787-05b3-45a6-899a-f13a1e511085 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:15.922477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb55787-05b3-45a6-899a-f13a1e511085 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:15.922749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb55787-05b3-45a6-899a-f13a1e511085 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:15.940164 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb55787-05b3-45a6-899a-f13a1e511085 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:15.945010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-ffb55787-05b3-45a6-899a-f13a1e511085 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:35:15.945010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:15.945010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:15.945010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:15.945010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:15.945010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:15.945010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:15.945010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:15.945010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:15.945010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:15.945010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:15.945010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:15.945010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:15.945010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:15.945010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:15.945010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:15.946409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:15.946409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:15.946409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:15.946409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:15.946409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:15.946409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:15.946409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:15.946409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:15.946409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:15.946409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:15.946409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:15.946409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:15.946409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:15.946409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:15.946409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:15.946409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:15.947627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:15.947627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:15.947627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:15.947627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:15.947627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:15.947627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:15.947627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:15.947627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:15.947627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:15.947627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:15.947627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:35:15.947627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:15.947627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 137/258] 10.222.0.36 () {56 vars in 1120 bytes} [Sun Oct 30 00:35:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 50 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:15.953429 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936295ae-474c-46a8-8eb1-486601195301 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:15.977714 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936295ae-474c-46a8-8eb1-486601195301 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:15.993758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52197197-5124-42e4-a147-e57b86e2e5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:15.994807 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52197197-5124-42e4-a147-e57b86e2e5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:15.995270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52197197-5124-42e4-a147-e57b86e2e5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:15.995736 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52197197-5124-42e4-a147-e57b86e2e5b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:15.996931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52197197-5124-42e4-a147-e57b86e2e5b1 admin admin] RBAC: Adding query filter params (name=member) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:15.997524 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52197197-5124-42e4-a147-e57b86e2e5b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:16.000443 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39eb5e-72ac-4a89-a364-8657e2582481 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:16.000679 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: WARNING py.warnings [None req-52197197-5124-42e4-a147-e57b86e2e5b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:35:16.000679 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: warnings.warn(msg) Oct 30 00:35:16.000679 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: Oct 30 00:35:16.001088 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39eb5e-72ac-4a89-a364-8657e2582481 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:16.001088 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39eb5e-72ac-4a89-a364-8657e2582481 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:16.002527 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52197197-5124-42e4-a147-e57b86e2e5b1 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:16.002768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 138/259] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:16.008206 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669f4a6-5aa8-4134-8ef7-02b55d57c2dd None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:16.008702 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669f4a6-5aa8-4134-8ef7-02b55d57c2dd None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:16.009130 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669f4a6-5aa8-4134-8ef7-02b55d57c2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:16.024187 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 122/260] 10.222.0.36 () {56 vars in 1122 bytes} [Sun Oct 30 00:35:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:16.026653 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52197197-5124-42e4-a147-e57b86e2e5b1 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:16.040888 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-1669f4a6-5aa8-4134-8ef7-02b55d57c2dd None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:16.042366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-1669f4a6-5aa8-4134-8ef7-02b55d57c2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:16.050263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52197197-5124-42e4-a147-e57b86e2e5b1 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:16.062292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 139/261] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 30 00:35:16.065928 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a94a0c-1e6e-4f46-9c65-3e9e3594b703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:16.065928 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a94a0c-1e6e-4f46-9c65-3e9e3594b703 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:16.065928 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a94a0c-1e6e-4f46-9c65-3e9e3594b703 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:16.065928 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a94a0c-1e6e-4f46-9c65-3e9e3594b703 admin admin] PATH_INFO: `/v3/groups/d04550585c1b41f493f4a776f88ce662` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:16.070552 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a8608-4010-4f87-9223-582efdda4e60 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:16.071334 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a8608-4010-4f87-9223-582efdda4e60 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:16.072057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a8608-4010-4f87-9223-582efdda4e60 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:16.075592 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 140/262] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:16.081579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a94a0c-1e6e-4f46-9c65-3e9e3594b703 admin admin] RBAC: Authorizing `identity:get_group(group_id=d04550585c1b41f493f4a776f88ce662)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:16.082246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175f81f-add0-4a66-99f3-0481a1201ac0 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:16.082626 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: WARNING py.warnings [None req-f2a94a0c-1e6e-4f46-9c65-3e9e3594b703 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:35:16.082626 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: warnings.warn(msg) Oct 30 00:35:16.082626 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: Oct 30 00:35:16.083529 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175f81f-add0-4a66-99f3-0481a1201ac0 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:16.083867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a94a0c-1e6e-4f46-9c65-3e9e3594b703 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:16.084618 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175f81f-add0-4a66-99f3-0481a1201ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:16.085577 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 123/263] 10.222.0.36 () {56 vars in 1201 bytes} [Sun Oct 30 00:35:16 2022] GET /identity/v3/groups/d04550585c1b41f493f4a776f88ce662 => generated 226 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:16.096120 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a94a0c-1e6e-4f46-9c65-3e9e3594b703 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:16.106862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a94a0c-1e6e-4f46-9c65-3e9e3594b703 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:16.120802 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d911570-e59b-4457-83c0-707bd11b1dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:16.121822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d911570-e59b-4457-83c0-707bd11b1dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:16.122306 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d911570-e59b-4457-83c0-707bd11b1dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:16.122780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d911570-e59b-4457-83c0-707bd11b1dc0 admin admin] PATH_INFO: `/v3/projects/72d02777c6694301959a12c02452adf3` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:16.124292 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d911570-e59b-4457-83c0-707bd11b1dc0 admin admin] RBAC: Authorizing `identity:get_project(project_id=72d02777c6694301959a12c02452adf3)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:16.126598 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d911570-e59b-4457-83c0-707bd11b1dc0 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:16.128515 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 124/264] 10.222.0.36 () {56 vars in 1207 bytes} [Sun Oct 30 00:35:16 2022] GET /identity/v3/projects/72d02777c6694301959a12c02452adf3 => generated 298 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:16.134448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d911570-e59b-4457-83c0-707bd11b1dc0 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:16.143752 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-7175f81f-add0-4a66-99f3-0481a1201ac0 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:16.145741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-7175f81f-add0-4a66-99f3-0481a1201ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:16.151693 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d911570-e59b-4457-83c0-707bd11b1dc0 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:16.162473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7690fb-37a3-4ae8-836e-2d2ae23c8f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:16.162887 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 141/265] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 30 00:35:16.163741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7690fb-37a3-4ae8-836e-2d2ae23c8f2e admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:16.164305 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7690fb-37a3-4ae8-836e-2d2ae23c8f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:16.164878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7690fb-37a3-4ae8-836e-2d2ae23c8f2e admin admin] PATH_INFO: `/v3/projects/72d02777c6694301959a12c02452adf3/groups/d04550585c1b41f493f4a776f88ce662/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:16.167075 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7690fb-37a3-4ae8-836e-2d2ae23c8f2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=72d02777c6694301959a12c02452adf3, group_id=d04550585c1b41f493f4a776f88ce662, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:16.168356 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: WARNING py.warnings [None req-fc7690fb-37a3-4ae8-836e-2d2ae23c8f2e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:35:16.168356 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: warnings.warn(msg) Oct 30 00:35:16.168356 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: Oct 30 00:35:16.170706 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7690fb-37a3-4ae8-836e-2d2ae23c8f2e admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:16.190394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 125/266] 10.222.0.36 () {58 vars in 1463 bytes} [Sun Oct 30 00:35:16 2022] PUT /identity/v3/projects/72d02777c6694301959a12c02452adf3/groups/d04550585c1b41f493f4a776f88ce662/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:35:16.504942 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175f81f-add0-4a66-99f3-0481a1201ac0 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:16.514989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175f81f-add0-4a66-99f3-0481a1201ac0 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:16.591548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7690fb-37a3-4ae8-836e-2d2ae23c8f2e admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:16.614304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef925c3-f330-48b6-847e-3fa9255d2416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:16.615251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef925c3-f330-48b6-847e-3fa9255d2416 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:16.615532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef925c3-f330-48b6-847e-3fa9255d2416 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:16.615811 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef925c3-f330-48b6-847e-3fa9255d2416 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:16.622378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:16.641625 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef925c3-f330-48b6-847e-3fa9255d2416 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:16.647252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-9ef925c3-f330-48b6-847e-3fa9255d2416 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:35:16.647252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:16.647252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:16.647252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:16.647252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:16.647252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:16.647252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:16.647252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:16.647252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:16.647252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:16.647252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:16.647252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:16.647252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:16.647252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:16.647252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:16.647252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:16.648187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:16.648187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:16.648187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:16.648187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:16.648187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:16.648187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:16.648187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:16.648187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:16.648187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:16.648187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:16.648187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:16.648187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:16.648187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:16.648187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:16.648187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:16.648187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:16.650528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:16.650528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:16.650528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:16.650528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:16.650528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:16.650528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:16.650528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:16.650528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:16.650528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:16.650528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:16.650528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:35:16.650528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:16.650528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 142/267] 10.222.0.36 () {56 vars in 1135 bytes} [Sun Oct 30 00:35:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:16.654931 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef925c3-f330-48b6-847e-3fa9255d2416 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:16.672014 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef925c3-f330-48b6-847e-3fa9255d2416 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:16.681360 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b6c4e-7512-4eec-9612-4523a3a73809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:16.681987 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b6c4e-7512-4eec-9612-4523a3a73809 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:16.682265 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b6c4e-7512-4eec-9612-4523a3a73809 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:16.682538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b6c4e-7512-4eec-9612-4523a3a73809 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:16.683428 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b6c4e-7512-4eec-9612-4523a3a73809 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:16.683674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b6c4e-7512-4eec-9612-4523a3a73809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:16.685852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: WARNING py.warnings [None req-404b6c4e-7512-4eec-9612-4523a3a73809 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:35:16.685852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: warnings.warn(msg) Oct 30 00:35:16.685852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: Oct 30 00:35:16.686789 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b6c4e-7512-4eec-9612-4523a3a73809 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:16.703772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 143/268] 10.222.0.36 () {56 vars in 1132 bytes} [Sun Oct 30 00:35:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:16.709488 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b6c4e-7512-4eec-9612-4523a3a73809 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:16.718505 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7690fb-37a3-4ae8-836e-2d2ae23c8f2e admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:16.728212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6bcb6-a779-4797-9785-6cf559fb23e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:16.728870 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6bcb6-a779-4797-9785-6cf559fb23e6 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:16.729202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6bcb6-a779-4797-9785-6cf559fb23e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:16.729474 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6bcb6-a779-4797-9785-6cf559fb23e6 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:16.732763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b6c4e-7512-4eec-9612-4523a3a73809 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:16.741060 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:16.743891 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bf295-3b7f-4f4a-8c99-a99b5c11aa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:16.744605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bf295-3b7f-4f4a-8c99-a99b5c11aa85 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:16.744884 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bf295-3b7f-4f4a-8c99-a99b5c11aa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:16.745159 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bf295-3b7f-4f4a-8c99-a99b5c11aa85 admin admin] PATH_INFO: `/v3/groups/d04550585c1b41f493f4a776f88ce662` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:16.749105 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598bf295-3b7f-4f4a-8c99-a99b5c11aa85 admin admin] RBAC: Authorizing `identity:get_group(group_id=d04550585c1b41f493f4a776f88ce662)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:16.749405 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6bcb6-a779-4797-9785-6cf559fb23e6 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:16.750153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: WARNING py.warnings [None req-598bf295-3b7f-4f4a-8c99-a99b5c11aa85 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:35:16.750153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: warnings.warn(msg) Oct 30 00:35:16.750153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: Oct 30 00:35:16.751390 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598bf295-3b7f-4f4a-8c99-a99b5c11aa85 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:16.753065 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 144/269] 10.222.0.36 () {56 vars in 1201 bytes} [Sun Oct 30 00:35:16 2022] GET /identity/v3/groups/d04550585c1b41f493f4a776f88ce662 => generated 226 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:16.754558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-1ac6bcb6-a779-4797-9785-6cf559fb23e6 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:35:16.754558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:16.754558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:16.754558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:16.754558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:16.754558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:16.754558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:16.754558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:16.754558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:16.754558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:16.754558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:16.754558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:16.754558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:16.754558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:16.754558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:16.754558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:16.756968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:16.756968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:16.756968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:16.756968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:16.756968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:16.756968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:16.756968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:16.756968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:16.756968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:16.756968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:16.756968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:16.756968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:16.756968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:16.756968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:16.756968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:16.756968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:16.759467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:16.759467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:16.759467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:16.759467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:16.759467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:16.759467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:16.759467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:16.759467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:16.759467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:16.759467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:16.759467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:35:16.759467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:16.759467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 126/270] 10.222.0.36 () {56 vars in 1120 bytes} [Sun Oct 30 00:35:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:35:16.763926 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bf295-3b7f-4f4a-8c99-a99b5c11aa85 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:16.764795 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6bcb6-a779-4797-9785-6cf559fb23e6 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:16.782065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6bcb6-a779-4797-9785-6cf559fb23e6 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:16.786341 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bf295-3b7f-4f4a-8c99-a99b5c11aa85 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:16.790927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08c04f-4f38-41ab-99de-b564a7a60a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:16.791684 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08c04f-4f38-41ab-99de-b564a7a60a4a admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:16.791966 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08c04f-4f38-41ab-99de-b564a7a60a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:16.792261 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08c04f-4f38-41ab-99de-b564a7a60a4a admin admin] PATH_INFO: `/v3/projects/72d02777c6694301959a12c02452adf3` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:16.793446 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08c04f-4f38-41ab-99de-b564a7a60a4a admin admin] RBAC: Authorizing `identity:get_project(project_id=72d02777c6694301959a12c02452adf3)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:16.795888 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08c04f-4f38-41ab-99de-b564a7a60a4a admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:16.798032 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 127/271] 10.222.0.36 () {56 vars in 1207 bytes} [Sun Oct 30 00:35:16 2022] GET /identity/v3/projects/72d02777c6694301959a12c02452adf3 => generated 298 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:16.803302 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08c04f-4f38-41ab-99de-b564a7a60a4a admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:16.806146 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa2d97-1efb-4d64-b457-70892e5764d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:16.806755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa2d97-1efb-4d64-b457-70892e5764d1 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:16.807034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa2d97-1efb-4d64-b457-70892e5764d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:16.807310 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa2d97-1efb-4d64-b457-70892e5764d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:16.807994 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa2d97-1efb-4d64-b457-70892e5764d1 admin admin] RBAC: Adding query filter params (name=member) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:16.808321 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa2d97-1efb-4d64-b457-70892e5764d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:16.810248 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa2d97-1efb-4d64-b457-70892e5764d1 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:16.818434 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08c04f-4f38-41ab-99de-b564a7a60a4a admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:16.824448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 145/272] 10.222.0.36 () {56 vars in 1122 bytes} [Sun Oct 30 00:35:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:16.826751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0c47f-6642-4098-af5a-01da4c33639e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:16.827654 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0c47f-6642-4098-af5a-01da4c33639e admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:16.827936 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0c47f-6642-4098-af5a-01da4c33639e admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:16.828233 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0c47f-6642-4098-af5a-01da4c33639e admin admin] PATH_INFO: `/v3/projects/72d02777c6694301959a12c02452adf3/groups/d04550585c1b41f493f4a776f88ce662/roles/af3529d63bae4650be7b8dc67b3ed527` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:16.830549 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d0c47f-6642-4098-af5a-01da4c33639e admin admin] RBAC: Authorizing `identity:create_grant(project_id=72d02777c6694301959a12c02452adf3, group_id=d04550585c1b41f493f4a776f88ce662, role_id=af3529d63bae4650be7b8dc67b3ed527)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:16.833558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d0c47f-6642-4098-af5a-01da4c33639e admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:16.841966 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa2d97-1efb-4d64-b457-70892e5764d1 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:16.854211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 128/273] 10.222.0.36 () {58 vars in 1463 bytes} [Sun Oct 30 00:35:16 2022] PUT /identity/v3/projects/72d02777c6694301959a12c02452adf3/groups/d04550585c1b41f493f4a776f88ce662/roles/af3529d63bae4650be7b8dc67b3ed527 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:35:16.861731 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa2d97-1efb-4d64-b457-70892e5764d1 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:16.958109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc457155-ee9e-4d87-8548-157e4132d3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:16.958997 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc457155-ee9e-4d87-8548-157e4132d3ba admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:16.959285 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc457155-ee9e-4d87-8548-157e4132d3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:16.959561 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc457155-ee9e-4d87-8548-157e4132d3ba admin admin] PATH_INFO: `/v3/groups/d04550585c1b41f493f4a776f88ce662` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:16.960729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc457155-ee9e-4d87-8548-157e4132d3ba admin admin] RBAC: Authorizing `identity:get_group(group_id=d04550585c1b41f493f4a776f88ce662)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:16.964517 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc457155-ee9e-4d87-8548-157e4132d3ba admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:16.966658 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 146/274] 10.222.0.36 () {56 vars in 1201 bytes} [Sun Oct 30 00:35:16 2022] GET /identity/v3/groups/d04550585c1b41f493f4a776f88ce662 => generated 226 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:16.972203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0c47f-6642-4098-af5a-01da4c33639e admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:16.989929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0c47f-6642-4098-af5a-01da4c33639e admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:16.998531 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f302334-74ba-432f-9eb7-7ced1264f93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:16.999206 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f302334-74ba-432f-9eb7-7ced1264f93c admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:16.999480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f302334-74ba-432f-9eb7-7ced1264f93c admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:16.999812 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f302334-74ba-432f-9eb7-7ced1264f93c admin admin] PATH_INFO: `/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:17.001052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f302334-74ba-432f-9eb7-7ced1264f93c admin admin] RBAC: Authorizing `identity:get_project(project_id=b9c6b20f7b7a4967ada2d4c9d25dd94a)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:17.003300 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f302334-74ba-432f-9eb7-7ced1264f93c admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:17.005114 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 129/275] 10.222.0.36 () {56 vars in 1207 bytes} [Sun Oct 30 00:35:16 2022] GET /identity/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a => generated 302 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:17.011380 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc457155-ee9e-4d87-8548-157e4132d3ba admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:17.026127 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc457155-ee9e-4d87-8548-157e4132d3ba admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:17.034904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f574536-3402-485f-bbcb-6cc88f070146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:17.035598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f574536-3402-485f-bbcb-6cc88f070146 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:17.035877 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f574536-3402-485f-bbcb-6cc88f070146 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:17.036220 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f574536-3402-485f-bbcb-6cc88f070146 admin admin] PATH_INFO: `/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a/groups/d04550585c1b41f493f4a776f88ce662/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:17.041403 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f574536-3402-485f-bbcb-6cc88f070146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9c6b20f7b7a4967ada2d4c9d25dd94a, group_id=d04550585c1b41f493f4a776f88ce662, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:17.042478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: WARNING py.warnings [None req-6f574536-3402-485f-bbcb-6cc88f070146 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:35:17.042478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: warnings.warn(msg) Oct 30 00:35:17.042478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: Oct 30 00:35:17.044556 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f574536-3402-485f-bbcb-6cc88f070146 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:17.065611 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 147/276] 10.222.0.36 () {58 vars in 1463 bytes} [Sun Oct 30 00:35:17 2022] PUT /identity/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a/groups/d04550585c1b41f493f4a776f88ce662/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:35:17.441799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3077392-57bf-4f47-91ae-3f2083aa617e None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:17.442168 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3077392-57bf-4f47-91ae-3f2083aa617e None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:17.442337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3077392-57bf-4f47-91ae-3f2083aa617e None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:17.443672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 130/277] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:17.450433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0aa95c-eb47-49f8-ac99-a2dd3ca14957 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:17.450699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0aa95c-eb47-49f8-ac99-a2dd3ca14957 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:17.450971 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0aa95c-eb47-49f8-ac99-a2dd3ca14957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:17.474385 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a97389-6925-4a6a-bafb-0353ac66284e None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:17.474709 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a97389-6925-4a6a-bafb-0353ac66284e None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:17.474915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a97389-6925-4a6a-bafb-0353ac66284e None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:17.476452 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 131/278] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:35:17.485938 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4cd2c-82fb-4dcf-908d-ed3b2de30507 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:17.485938 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4cd2c-82fb-4dcf-908d-ed3b2de30507 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:17.485938 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4cd2c-82fb-4dcf-908d-ed3b2de30507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:17.486495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-7f0aa95c-eb47-49f8-ac99-a2dd3ca14957 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:17.486495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-7f0aa95c-eb47-49f8-ac99-a2dd3ca14957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:17.508617 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-e8a4cd2c-82fb-4dcf-908d-ed3b2de30507 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:17.510877 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e8a4cd2c-82fb-4dcf-908d-ed3b2de30507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:17.595772 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 132/279] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:35:17.600737 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 148/280] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 30 00:35:18.162431 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4cd2c-82fb-4dcf-908d-ed3b2de30507 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:18.174179 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4cd2c-82fb-4dcf-908d-ed3b2de30507 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:18.181413 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0aa95c-eb47-49f8-ac99-a2dd3ca14957 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:18.183139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b48f3-73fd-4e25-90ac-1b032ca2ca1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:18.184125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b48f3-73fd-4e25-90ac-1b032ca2ca1f admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:18.184668 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b48f3-73fd-4e25-90ac-1b032ca2ca1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:18.185165 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b48f3-73fd-4e25-90ac-1b032ca2ca1f admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:18.204328 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b48f3-73fd-4e25-90ac-1b032ca2ca1f admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:18.210058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-7d7b48f3-73fd-4e25-90ac-1b032ca2ca1f admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:35:18.210058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:18.210058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:18.210058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:18.210058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:18.210058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:18.210058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:18.210058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:18.210058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:18.210058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:18.210058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:18.210058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:18.210058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:18.210058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:18.210058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:18.210058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:18.210994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:18.210994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:18.210994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:18.210994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:18.210994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:18.210994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:18.210994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:18.210994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:18.210994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:18.210994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:18.210994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:18.210994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:18.210994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:18.210994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:18.210994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:18.210994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:18.216777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:18.216777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:18.216777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:18.216777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:18.216777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:18.216777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:18.216777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:18.216777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:18.216777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:18.216777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:18.216777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:35:18.216777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:18.216777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 133/281] 10.222.0.36 () {56 vars in 1135 bytes} [Sun Oct 30 00:35:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:18.216777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58745488-d7f3-4e39-8ce0-3267f2de655c None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:18.217728 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58745488-d7f3-4e39-8ce0-3267f2de655c None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:18.217728 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58745488-d7f3-4e39-8ce0-3267f2de655c None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:18.217728 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 134/282] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:18.223219 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853528f-a8c7-4d71-9630-6f28203f3c21 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:18.223219 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853528f-a8c7-4d71-9630-6f28203f3c21 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:18.223219 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853528f-a8c7-4d71-9630-6f28203f3c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:18.251745 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-1853528f-a8c7-4d71-9630-6f28203f3c21 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:18.253009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-1853528f-a8c7-4d71-9630-6f28203f3c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:18.262829 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:18.264469 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 135/283] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 30 00:35:18.264774 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853528f-a8c7-4d71-9630-6f28203f3c21 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:18.282780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853528f-a8c7-4d71-9630-6f28203f3c21 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:18.296513 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4043c7-b068-4084-80ea-61ef911c3826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:18.297183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4043c7-b068-4084-80ea-61ef911c3826 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:18.297824 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4043c7-b068-4084-80ea-61ef911c3826 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:18.297824 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4043c7-b068-4084-80ea-61ef911c3826 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:18.298414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4043c7-b068-4084-80ea-61ef911c3826 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:18.298722 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4043c7-b068-4084-80ea-61ef911c3826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:18.300691 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4043c7-b068-4084-80ea-61ef911c3826 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:18.322004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 136/284] 10.222.0.36 () {56 vars in 1132 bytes} [Sun Oct 30 00:35:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:18.332624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0aa95c-eb47-49f8-ac99-a2dd3ca14957 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:18.346111 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4043c7-b068-4084-80ea-61ef911c3826 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:18.368022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4043c7-b068-4084-80ea-61ef911c3826 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:18.378230 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f236d-365e-44a5-989c-0b9196401e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:18.379242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f236d-365e-44a5-989c-0b9196401e6d None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:18.379712 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f236d-365e-44a5-989c-0b9196401e6d None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:18.380208 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f236d-365e-44a5-989c-0b9196401e6d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:18.393827 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f236d-365e-44a5-989c-0b9196401e6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:18.400243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-9d0f236d-365e-44a5-989c-0b9196401e6d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:35:18.400243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:18.400243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:18.400243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:18.400243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:18.400243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:18.400243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:18.400243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:18.400243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:18.400243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:18.400243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:18.400243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:18.400243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:18.400243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:18.400243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:18.400243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:18.401311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:18.401311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:18.401311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:18.401311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:18.401311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:18.401311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:18.401311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:18.401311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:18.401311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:18.401311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:18.401311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:18.401311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:18.401311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:18.401311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:18.401311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:18.401311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:18.402104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:18.402104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:18.402104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:18.402104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:18.402104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:18.402104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:18.402104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:18.402104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:18.402104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:18.402104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:18.402104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 30 00:35:18.402104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:18.403638 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85cd600-d661-4faf-a882-9cfede834b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:18.404332 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85cd600-d661-4faf-a882-9cfede834b85 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:18.404616 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85cd600-d661-4faf-a882-9cfede834b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:18.404896 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85cd600-d661-4faf-a882-9cfede834b85 admin admin] PATH_INFO: `/v3/groups/d04550585c1b41f493f4a776f88ce662` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:18.408870 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85cd600-d661-4faf-a882-9cfede834b85 admin admin] RBAC: Authorizing `identity:get_group(group_id=d04550585c1b41f493f4a776f88ce662)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:18.409191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 149/285] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:18.410796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85cd600-d661-4faf-a882-9cfede834b85 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:18.412862 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f236d-365e-44a5-989c-0b9196401e6d None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:18.413121 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 137/286] 10.222.0.36 () {56 vars in 1201 bytes} [Sun Oct 30 00:35:18 2022] GET /identity/v3/groups/d04550585c1b41f493f4a776f88ce662 => generated 226 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:18.426124 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85cd600-d661-4faf-a882-9cfede834b85 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:18.447321 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85cd600-d661-4faf-a882-9cfede834b85 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:18.468837 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c1a89-e554-45b8-9172-68fd4b6a9132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:18.469503 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c1a89-e554-45b8-9172-68fd4b6a9132 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:18.469787 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c1a89-e554-45b8-9172-68fd4b6a9132 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:18.470066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c1a89-e554-45b8-9172-68fd4b6a9132 admin admin] PATH_INFO: `/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:18.471322 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c1a89-e554-45b8-9172-68fd4b6a9132 admin admin] RBAC: Authorizing `identity:get_project(project_id=b9c6b20f7b7a4967ada2d4c9d25dd94a)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:18.473444 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c1a89-e554-45b8-9172-68fd4b6a9132 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:18.478730 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 138/287] 10.222.0.36 () {56 vars in 1207 bytes} [Sun Oct 30 00:35:18 2022] GET /identity/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a => generated 302 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:18.481589 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c1a89-e554-45b8-9172-68fd4b6a9132 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:18.513300 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c1a89-e554-45b8-9172-68fd4b6a9132 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:18.532616 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc66a4-bdf5-44e1-9a4d-7e6c9a98ad8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:18.535857 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc66a4-bdf5-44e1-9a4d-7e6c9a98ad8b admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:18.536516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc66a4-bdf5-44e1-9a4d-7e6c9a98ad8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:18.537695 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc66a4-bdf5-44e1-9a4d-7e6c9a98ad8b admin admin] PATH_INFO: `/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a/groups/d04550585c1b41f493f4a776f88ce662/roles/af3529d63bae4650be7b8dc67b3ed527` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:18.544027 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc66a4-bdf5-44e1-9a4d-7e6c9a98ad8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9c6b20f7b7a4967ada2d4c9d25dd94a, group_id=d04550585c1b41f493f4a776f88ce662, role_id=af3529d63bae4650be7b8dc67b3ed527)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:18.556275 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc66a4-bdf5-44e1-9a4d-7e6c9a98ad8b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:18.589476 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 139/288] 10.222.0.36 () {58 vars in 1463 bytes} [Sun Oct 30 00:35:18 2022] PUT /identity/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a/groups/d04550585c1b41f493f4a776f88ce662/roles/af3529d63bae4650be7b8dc67b3ed527 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:35:18.641601 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f236d-365e-44a5-989c-0b9196401e6d None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:18.650622 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc66a4-bdf5-44e1-9a4d-7e6c9a98ad8b admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:18.672698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:18.713237 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5897c-0fda-45b3-b5b0-aaafac624717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:18.713916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5897c-0fda-45b3-b5b0-aaafac624717 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:18.714200 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5897c-0fda-45b3-b5b0-aaafac624717 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:18.714524 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5897c-0fda-45b3-b5b0-aaafac624717 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:18.715432 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5897c-0fda-45b3-b5b0-aaafac624717 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:18.715661 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5897c-0fda-45b3-b5b0-aaafac624717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:18.717234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5897c-0fda-45b3-b5b0-aaafac624717 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:18.728540 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 150/289] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:35:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:18.734145 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5897c-0fda-45b3-b5b0-aaafac624717 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:18.737339 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc66a4-bdf5-44e1-9a4d-7e6c9a98ad8b admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:18.742919 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73bd55-1443-4e65-967f-fbef407f9290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:18.743811 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73bd55-1443-4e65-967f-fbef407f9290 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:18.744261 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73bd55-1443-4e65-967f-fbef407f9290 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:18.744670 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73bd55-1443-4e65-967f-fbef407f9290 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:18.754142 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:18.759540 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73bd55-1443-4e65-967f-fbef407f9290 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:18.766430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-4c73bd55-1443-4e65-967f-fbef407f9290 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:35:18.766430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:18.766430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:18.766430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:18.766430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:18.766430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:18.766430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:18.766430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:18.766430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:18.766430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:18.766430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:18.766430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:18.766430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:18.766430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:18.766430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:18.766430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:18.767402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:18.767402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:18.767402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:18.767402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:18.767402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:18.767402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:18.767402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:18.767402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:18.767402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:18.767402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:18.767402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:18.767402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:18.767402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:18.767402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:18.767402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:18.767402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:18.768521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:18.768521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:18.768521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:18.768521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:18.768521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:18.768521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:18.768521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:18.768521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:18.768521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:18.768521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:18.768521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:35:18.768521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:18.771021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 140/290] 10.222.0.36 () {56 vars in 1117 bytes} [Sun Oct 30 00:35:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 122 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:18.777371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73bd55-1443-4e65-967f-fbef407f9290 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:18.788975 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73bd55-1443-4e65-967f-fbef407f9290 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:18.794811 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100164a-4c20-499b-86ed-c2cef85bf669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:18.795787 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100164a-4c20-499b-86ed-c2cef85bf669 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:18.796993 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100164a-4c20-499b-86ed-c2cef85bf669 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:18.797501 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100164a-4c20-499b-86ed-c2cef85bf669 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:18.798460 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100164a-4c20-499b-86ed-c2cef85bf669 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:18.798982 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100164a-4c20-499b-86ed-c2cef85bf669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:18.800905 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100164a-4c20-499b-86ed-c2cef85bf669 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:18.810551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 141/291] 10.222.0.36 () {56 vars in 1120 bytes} [Sun Oct 30 00:35:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:18.816514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100164a-4c20-499b-86ed-c2cef85bf669 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:18.828689 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100164a-4c20-499b-86ed-c2cef85bf669 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:18.833633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35de82-469d-4449-bb30-6e621170ab53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:18.834591 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35de82-469d-4449-bb30-6e621170ab53 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:18.835173 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35de82-469d-4449-bb30-6e621170ab53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:18.835672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35de82-469d-4449-bb30-6e621170ab53 admin admin] PATH_INFO: `/v3/groups/38243fea584d42b7b0ac1d5f5cb63238` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:18.842245 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35de82-469d-4449-bb30-6e621170ab53 admin admin] RBAC: Authorizing `identity:get_group(group_id=38243fea584d42b7b0ac1d5f5cb63238)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:18.844658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35de82-469d-4449-bb30-6e621170ab53 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:18.847007 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 142/292] 10.222.0.36 () {56 vars in 1201 bytes} [Sun Oct 30 00:35:18 2022] GET /identity/v3/groups/38243fea584d42b7b0ac1d5f5cb63238 => generated 229 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:18.853355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35de82-469d-4449-bb30-6e621170ab53 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:18.864863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35de82-469d-4449-bb30-6e621170ab53 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:18.870555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152386db-8b08-44be-a63d-a9345337d701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:18.872185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152386db-8b08-44be-a63d-a9345337d701 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:18.872723 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152386db-8b08-44be-a63d-a9345337d701 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:18.873874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152386db-8b08-44be-a63d-a9345337d701 admin admin] PATH_INFO: `/v3/projects/3c2609b991ac4a86bca190e144b13290` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:18.875454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152386db-8b08-44be-a63d-a9345337d701 admin admin] RBAC: Authorizing `identity:get_project(project_id=3c2609b991ac4a86bca190e144b13290)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:18.875812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5897c-0fda-45b3-b5b0-aaafac624717 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:18.877940 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152386db-8b08-44be-a63d-a9345337d701 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:18.886282 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 143/293] 10.222.0.36 () {56 vars in 1207 bytes} [Sun Oct 30 00:35:18 2022] GET /identity/v3/projects/3c2609b991ac4a86bca190e144b13290 => generated 344 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:18.887632 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152386db-8b08-44be-a63d-a9345337d701 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:18.898906 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152386db-8b08-44be-a63d-a9345337d701 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:18.904626 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218131d5-9a26-4390-bf30-8313cba6e32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:18.905668 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218131d5-9a26-4390-bf30-8313cba6e32e admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:18.906162 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218131d5-9a26-4390-bf30-8313cba6e32e admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:18.906674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218131d5-9a26-4390-bf30-8313cba6e32e admin admin] PATH_INFO: `/v3/projects/3c2609b991ac4a86bca190e144b13290/groups/38243fea584d42b7b0ac1d5f5cb63238/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:18.909083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218131d5-9a26-4390-bf30-8313cba6e32e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c2609b991ac4a86bca190e144b13290, group_id=38243fea584d42b7b0ac1d5f5cb63238, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:18.912506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218131d5-9a26-4390-bf30-8313cba6e32e admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:18.927579 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201bcd62-062b-43dd-ac3e-5ee857447e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:18.928219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bcd62-062b-43dd-ac3e-5ee857447e33 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:18.928583 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bcd62-062b-43dd-ac3e-5ee857447e33 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:18.928958 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bcd62-062b-43dd-ac3e-5ee857447e33 None admin] PATH_INFO: `/v3/users/d539a888f6084a2d9afa689aed43bbff` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:18.930649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201bcd62-062b-43dd-ac3e-5ee857447e33 None admin] RBAC: Authorizing `identity:get_user(user_id=d539a888f6084a2d9afa689aed43bbff)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:18.932615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201bcd62-062b-43dd-ac3e-5ee857447e33 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:18.938287 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 151/294] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:35:18 2022] GET /identity/v3/users/d539a888f6084a2d9afa689aed43bbff => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:18.938483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 144/295] 10.222.0.36 () {58 vars in 1463 bytes} [Sun Oct 30 00:35:18 2022] PUT /identity/v3/projects/3c2609b991ac4a86bca190e144b13290/groups/38243fea584d42b7b0ac1d5f5cb63238/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:35:18.940780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201bcd62-062b-43dd-ac3e-5ee857447e33 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:19.001332 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:19.092705 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201bcd62-062b-43dd-ac3e-5ee857447e33 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:19.141303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363f60f-c954-4e82-b249-5c1db638041c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:19.141927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363f60f-c954-4e82-b249-5c1db638041c None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:19.142205 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363f60f-c954-4e82-b249-5c1db638041c None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:19.142484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363f60f-c954-4e82-b249-5c1db638041c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:19.143222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363f60f-c954-4e82-b249-5c1db638041c None admin] RBAC: Adding query filter params (role.id=2e58f1f4fb154236982461a05ee16601, user.id=d539a888f6084a2d9afa689aed43bbff, scope.system=all) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:19.143533 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363f60f-c954-4e82-b249-5c1db638041c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:19.145035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363f60f-c954-4e82-b249-5c1db638041c None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:19.161630 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 152/296] 10.222.0.36 () {56 vars in 1308 bytes} [Sun Oct 30 00:35:18 2022] GET /identity/v3/role_assignments?user.id=d539a888f6084a2d9afa689aed43bbff&scope.system=all&role.id=2e58f1f4fb154236982461a05ee16601 => generated 510 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:19.766664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9ad5e-ec23-40d3-9a7b-d82c5cdcc1ff None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:19.767201 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9ad5e-ec23-40d3-9a7b-d82c5cdcc1ff None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:19.767485 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9ad5e-ec23-40d3-9a7b-d82c5cdcc1ff None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:19.769848 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 145/297] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:19.777308 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88032070-2007-449b-84cd-3db769ccba46 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:19.777838 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88032070-2007-449b-84cd-3db769ccba46 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:19.778184 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88032070-2007-449b-84cd-3db769ccba46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:19.812635 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-88032070-2007-449b-84cd-3db769ccba46 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:19.814397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-88032070-2007-449b-84cd-3db769ccba46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:19.982316 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 153/298] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:20.413401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9ad5e-ec23-40d3-9a7b-d82c5cdcc1ff None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:20.538565 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243f8fe-b9e5-4a99-b2af-50519d15edde None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:20.539384 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243f8fe-b9e5-4a99-b2af-50519d15edde None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:20.540473 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243f8fe-b9e5-4a99-b2af-50519d15edde None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:20.542304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 154/299] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:20.553409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d709c0-f793-4544-8220-8b0f63c76295 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:20.553970 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d709c0-f793-4544-8220-8b0f63c76295 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:20.554440 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d709c0-f793-4544-8220-8b0f63c76295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:20.560895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9ad5e-ec23-40d3-9a7b-d82c5cdcc1ff None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:20.591081 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-d2d709c0-f793-4544-8220-8b0f63c76295 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:20.593398 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d2d709c0-f793-4544-8220-8b0f63c76295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:20.618388 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13ffa8-9e00-4c0e-a5ac-429a49f71024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:20.619716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13ffa8-9e00-4c0e-a5ac-429a49f71024 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:20.620757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13ffa8-9e00-4c0e-a5ac-429a49f71024 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:20.621255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13ffa8-9e00-4c0e-a5ac-429a49f71024 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:20.640339 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13ffa8-9e00-4c0e-a5ac-429a49f71024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:20.645061 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:20.654117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-2d13ffa8-9e00-4c0e-a5ac-429a49f71024 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:35:20.654117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:20.654117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:20.654117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:20.654117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:20.654117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:20.654117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:20.654117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:20.654117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:20.654117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:20.654117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:20.654117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:20.654117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:20.654117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:20.654117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:20.654117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:20.655068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:20.655068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:20.655068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:20.655068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:20.655068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:20.655068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:20.655068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:20.655068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:20.655068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:20.655068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:20.655068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:20.655068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:20.655068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:20.655068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:20.655068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:20.655068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:20.655867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:20.655867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:20.655867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:20.655867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:20.655867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:20.655867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:20.655867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:20.655867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:20.655867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:20.655867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:20.655867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:35:20.655867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:20.662859 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 146/300] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:20.686949 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13ffa8-9e00-4c0e-a5ac-429a49f71024 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:20.733748 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:20.747015 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 155/301] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:20.835410 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13ffa8-9e00-4c0e-a5ac-429a49f71024 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:20.862363 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d5b3c-0613-42ce-87ba-163f8edddee7 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:20.863393 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d5b3c-0613-42ce-87ba-163f8edddee7 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:20.863863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d5b3c-0613-42ce-87ba-163f8edddee7 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:20.865452 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 156/302] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:35:20.882488 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb922ee-a0d9-4dd9-bb89-9d51a2c3947c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:20.883702 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb922ee-a0d9-4dd9-bb89-9d51a2c3947c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:20.884959 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb922ee-a0d9-4dd9-bb89-9d51a2c3947c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:20.922413 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-9eb922ee-a0d9-4dd9-bb89-9d51a2c3947c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:20.925466 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-9eb922ee-a0d9-4dd9-bb89-9d51a2c3947c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:20.933560 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73720c1-c778-431e-aa9e-759fd33f7e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:20.934570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73720c1-c778-431e-aa9e-759fd33f7e2b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:20.935045 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73720c1-c778-431e-aa9e-759fd33f7e2b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:20.935509 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73720c1-c778-431e-aa9e-759fd33f7e2b None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:20.936428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73720c1-c778-431e-aa9e-759fd33f7e2b None admin] RBAC: Adding query filter params (name=member) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:20.938701 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73720c1-c778-431e-aa9e-759fd33f7e2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:20.940370 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73720c1-c778-431e-aa9e-759fd33f7e2b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:20.963278 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 147/303] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:35:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:20.980309 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73720c1-c778-431e-aa9e-759fd33f7e2b None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:21.075359 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 157/304] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:21.111976 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:21.127576 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73720c1-c778-431e-aa9e-759fd33f7e2b None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:21.157498 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb922ee-a0d9-4dd9-bb89-9d51a2c3947c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:21.165684 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20ed15-2ce6-4e7f-9a0e-332c5b1be81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:21.166298 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20ed15-2ce6-4e7f-9a0e-332c5b1be81a None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:21.166579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20ed15-2ce6-4e7f-9a0e-332c5b1be81a None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:21.166851 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20ed15-2ce6-4e7f-9a0e-332c5b1be81a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:21.279800 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20ed15-2ce6-4e7f-9a0e-332c5b1be81a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:21.281247 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20ed15-2ce6-4e7f-9a0e-332c5b1be81a None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:21.332461 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:21.345193 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb922ee-a0d9-4dd9-bb89-9d51a2c3947c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:21.388817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:21.409906 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 148/305] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:35:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:35:21.416464 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04415e3-731c-496d-8058-24f6a3a70092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:21.416848 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20ed15-2ce6-4e7f-9a0e-332c5b1be81a None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:21.417266 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04415e3-731c-496d-8058-24f6a3a70092 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:21.417620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04415e3-731c-496d-8058-24f6a3a70092 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:21.418025 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04415e3-731c-496d-8058-24f6a3a70092 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:21.443425 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04415e3-731c-496d-8058-24f6a3a70092 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:21.450043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-f04415e3-731c-496d-8058-24f6a3a70092 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:35:21.450043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:21.450043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:21.450043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:21.450043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:21.450043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:21.450043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:21.450043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:21.450043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:21.450043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:21.450043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:21.450043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:21.450043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:21.450043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:21.450043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:21.450043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:21.451006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:21.451006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:21.451006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:21.451006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:21.451006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:21.451006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:21.451006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:21.451006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:21.451006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:21.451006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:21.451006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:21.451006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:21.451006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:21.451006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:21.451006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:21.451006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:21.452234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:21.452234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:21.452234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:21.452234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:21.452234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:21.452234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:21.452234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:21.452234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:21.452234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:21.452234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:21.452234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:35:21.452234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:21.452234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 158/306] 10.222.0.36 () {56 vars in 1114 bytes} [Sun Oct 30 00:35:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:35:21.462354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04415e3-731c-496d-8058-24f6a3a70092 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:21.611709 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20ed15-2ce6-4e7f-9a0e-332c5b1be81a None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:21.648204 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04415e3-731c-496d-8058-24f6a3a70092 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:21.666717 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66fce8-3a42-4f1a-96ac-dc164b689705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:21.667519 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66fce8-3a42-4f1a-96ac-dc164b689705 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:21.668057 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66fce8-3a42-4f1a-96ac-dc164b689705 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:21.668448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66fce8-3a42-4f1a-96ac-dc164b689705 None admin] PATH_INFO: `/v3/projects/72d02777c6694301959a12c02452adf3` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:21.669856 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66fce8-3a42-4f1a-96ac-dc164b689705 None admin] RBAC: Authorizing `identity:get_project(project_id=72d02777c6694301959a12c02452adf3)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:21.671511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66fce8-3a42-4f1a-96ac-dc164b689705 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:21.673890 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 149/307] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:35:21 2022] GET /identity/v3/projects/72d02777c6694301959a12c02452adf3 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:21.675231 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66fce8-3a42-4f1a-96ac-dc164b689705 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:21.708812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c03ec2-3a19-43aa-8660-5a1bd72359e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:21.709450 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c03ec2-3a19-43aa-8660-5a1bd72359e2 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:21.709739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c03ec2-3a19-43aa-8660-5a1bd72359e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:21.710069 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c03ec2-3a19-43aa-8660-5a1bd72359e2 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:21.710700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c03ec2-3a19-43aa-8660-5a1bd72359e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:21.711024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c03ec2-3a19-43aa-8660-5a1bd72359e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:21.713063 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c03ec2-3a19-43aa-8660-5a1bd72359e2 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:21.725381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 159/308] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:21.725772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c03ec2-3a19-43aa-8660-5a1bd72359e2 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:21.745559 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:21.786465 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:21.853462 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66fce8-3a42-4f1a-96ac-dc164b689705 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:21.881733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c03ec2-3a19-43aa-8660-5a1bd72359e2 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:21.905018 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cb7c9-d547-4603-a390-3ca431968e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:21.905018 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cb7c9-d547-4603-a390-3ca431968e34 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:21.905310 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cb7c9-d547-4603-a390-3ca431968e34 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:21.905641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cb7c9-d547-4603-a390-3ca431968e34 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:21.929123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609cb7c9-d547-4603-a390-3ca431968e34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:21.933756 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557988c-27e8-4aaf-ab3a-c33e43d97387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:21.934407 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557988c-27e8-4aaf-ab3a-c33e43d97387 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:21.934704 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557988c-27e8-4aaf-ab3a-c33e43d97387 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:21.934953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557988c-27e8-4aaf-ab3a-c33e43d97387 None admin] PATH_INFO: `/v3/groups/d04550585c1b41f493f4a776f88ce662` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:21.936102 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557988c-27e8-4aaf-ab3a-c33e43d97387 None admin] RBAC: Authorizing `identity:get_group(group_id=d04550585c1b41f493f4a776f88ce662)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:21.937571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557988c-27e8-4aaf-ab3a-c33e43d97387 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:21.939492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 160/309] 10.222.0.36 () {56 vars in 1180 bytes} [Sun Oct 30 00:35:21 2022] GET /identity/v3/groups/d04550585c1b41f493f4a776f88ce662 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:21.940379 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557988c-27e8-4aaf-ab3a-c33e43d97387 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:21.940853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-609cb7c9-d547-4603-a390-3ca431968e34 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:35:21.940853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:21.940853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:21.940853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:21.940853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:21.940853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:21.940853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:21.940853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:21.940853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:21.940853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:21.940853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:21.940853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:21.940853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:21.940853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:21.940853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:21.940853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:21.942609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:21.942609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:21.942609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:21.942609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:21.942609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:21.942609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:21.942609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:21.942609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:21.942609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:21.942609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:21.942609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:21.942609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:21.942609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:21.942609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:21.942609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:21.942609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:21.944257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:21.944257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:21.944257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:21.944257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:21.944257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:21.944257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:21.944257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:21.944257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:21.944257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:21.944257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:21.944257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 30 00:35:21.944257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:21.951109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 150/310] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:21.951109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbcd6d-47dc-4fdf-b6fc-9187b7af2866 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:21.951109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbcd6d-47dc-4fdf-b6fc-9187b7af2866 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:21.951109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbcd6d-47dc-4fdf-b6fc-9187b7af2866 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:21.951109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 151/311] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:21.951109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbcd6d-47dc-4fdf-b6fc-9187b7af2866 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:22.065280 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:22.090456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557988c-27e8-4aaf-ab3a-c33e43d97387 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:22.103670 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbcd6d-47dc-4fdf-b6fc-9187b7af2866 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:22.130089 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:22.139077 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6f556-bf66-4527-8ff4-9a7cc5457d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:22.139707 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6f556-bf66-4527-8ff4-9a7cc5457d02 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:22.139986 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6f556-bf66-4527-8ff4-9a7cc5457d02 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:22.140282 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6f556-bf66-4527-8ff4-9a7cc5457d02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:22.167720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a164a-9820-4260-9e3a-e88464e736b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:22.170412 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a164a-9820-4260-9e3a-e88464e736b1 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:22.170412 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a164a-9820-4260-9e3a-e88464e736b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:22.170412 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a164a-9820-4260-9e3a-e88464e736b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:22.170412 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a164a-9820-4260-9e3a-e88464e736b1 None admin] RBAC: Adding query filter params (group.id=d04550585c1b41f493f4a776f88ce662, role.id=2e0593ed85a14197a0eaf37135a01787, scope.project.id=72d02777c6694301959a12c02452adf3) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:22.170412 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a164a-9820-4260-9e3a-e88464e736b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:22.172268 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a164a-9820-4260-9e3a-e88464e736b1 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:22.181263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 152/312] 10.222.0.36 () {56 vars in 1376 bytes} [Sun Oct 30 00:35:21 2022] GET /identity/v3/role_assignments?group.id=d04550585c1b41f493f4a776f88ce662&scope.project.id=72d02777c6694301959a12c02452adf3&role.id=2e0593ed85a14197a0eaf37135a01787 => generated 611 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:22.181974 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a164a-9820-4260-9e3a-e88464e736b1 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:22.242978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d6f556-bf66-4527-8ff4-9a7cc5457d02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:22.244889 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d6f556-bf66-4527-8ff4-9a7cc5457d02 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:22.324690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a164a-9820-4260-9e3a-e88464e736b1 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:22.374010 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d5ce6-ad6f-4bf7-abde-54659b045e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:22.374644 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d5ce6-ad6f-4bf7-abde-54659b045e13 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:22.374985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d5ce6-ad6f-4bf7-abde-54659b045e13 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:22.375264 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d5ce6-ad6f-4bf7-abde-54659b045e13 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:22.376010 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d5ce6-ad6f-4bf7-abde-54659b045e13 None admin] RBAC: Adding query filter params (name=member) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:22.376351 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d5ce6-ad6f-4bf7-abde-54659b045e13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:22.377842 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d5ce6-ad6f-4bf7-abde-54659b045e13 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:22.400070 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 153/313] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:35:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:22.405467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfc133-1172-4070-b877-9730c8c0ccb1 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:22.405816 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfc133-1172-4070-b877-9730c8c0ccb1 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:22.406145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfc133-1172-4070-b877-9730c8c0ccb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:22.412757 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:22.434625 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-bbdfc133-1172-4070-b877-9730c8c0ccb1 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:22.435836 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-bbdfc133-1172-4070-b877-9730c8c0ccb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:22.440523 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 161/314] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:35:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:35:22.441697 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6f556-bf66-4527-8ff4-9a7cc5457d02 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:22.485514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:22.535004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 154/315] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:22.536643 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfc133-1172-4070-b877-9730c8c0ccb1 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:22.581652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6f556-bf66-4527-8ff4-9a7cc5457d02 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:22.628293 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa1db0-2ed9-4233-90a4-4e644096e2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:22.628908 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa1db0-2ed9-4233-90a4-4e644096e2d5 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:22.629183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa1db0-2ed9-4233-90a4-4e644096e2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:22.629452 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa1db0-2ed9-4233-90a4-4e644096e2d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:22.661522 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfc133-1172-4070-b877-9730c8c0ccb1 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:22.696339 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:22.723754 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa1db0-2ed9-4233-90a4-4e644096e2d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:22.725320 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be6a56-36f7-4af3-b786-605153ef9b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:22.725452 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa1db0-2ed9-4233-90a4-4e644096e2d5 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:22.725976 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be6a56-36f7-4af3-b786-605153ef9b6d None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:22.726255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be6a56-36f7-4af3-b786-605153ef9b6d None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:22.726527 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be6a56-36f7-4af3-b786-605153ef9b6d None admin] PATH_INFO: `/v3/projects/72d02777c6694301959a12c02452adf3` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:22.728704 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be6a56-36f7-4af3-b786-605153ef9b6d None admin] RBAC: Authorizing `identity:get_project(project_id=72d02777c6694301959a12c02452adf3)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:22.732280 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be6a56-36f7-4af3-b786-605153ef9b6d None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:22.733999 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 155/316] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:35:22 2022] GET /identity/v3/projects/72d02777c6694301959a12c02452adf3 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:22.735264 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1eee6-3b2f-43fd-87e2-925a0fbd51fb None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:22.735555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1eee6-3b2f-43fd-87e2-925a0fbd51fb None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:22.735826 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1eee6-3b2f-43fd-87e2-925a0fbd51fb None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:22.737951 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 156/317] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:22.741011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1eee6-3b2f-43fd-87e2-925a0fbd51fb None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:22.804364 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:22.861728 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 162/318] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:35:22.865993 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f001a-9659-48d2-b30b-9b6da4bded9f None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:22.865993 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f001a-9659-48d2-b30b-9b6da4bded9f None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:22.865993 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f001a-9659-48d2-b30b-9b6da4bded9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:22.882191 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1eee6-3b2f-43fd-87e2-925a0fbd51fb None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:22.887813 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-3e0f001a-9659-48d2-b30b-9b6da4bded9f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:22.889334 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-3e0f001a-9659-48d2-b30b-9b6da4bded9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:22.927742 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8310d-77c6-4107-96ca-1620d1935002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:22.928372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8310d-77c6-4107-96ca-1620d1935002 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:22.928656 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8310d-77c6-4107-96ca-1620d1935002 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:22.928929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8310d-77c6-4107-96ca-1620d1935002 None admin] PATH_INFO: `/v3/groups/d04550585c1b41f493f4a776f88ce662` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:22.930174 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8310d-77c6-4107-96ca-1620d1935002 None admin] RBAC: Authorizing `identity:get_group(group_id=d04550585c1b41f493f4a776f88ce662)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:22.931642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8310d-77c6-4107-96ca-1620d1935002 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:22.933307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 157/319] 10.222.0.36 () {56 vars in 1180 bytes} [Sun Oct 30 00:35:22 2022] GET /identity/v3/groups/d04550585c1b41f493f4a776f88ce662 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:22.934172 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8310d-77c6-4107-96ca-1620d1935002 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:22.988349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 163/320] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:22.989527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f001a-9659-48d2-b30b-9b6da4bded9f None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:23.002093 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:23.074188 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8310d-77c6-4107-96ca-1620d1935002 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:23.084990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:23.119081 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981c3cc-383a-4f22-bd7d-b6405148a439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:23.119730 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981c3cc-383a-4f22-bd7d-b6405148a439 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:23.120009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981c3cc-383a-4f22-bd7d-b6405148a439 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:23.120341 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981c3cc-383a-4f22-bd7d-b6405148a439 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:23.129945 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f001a-9659-48d2-b30b-9b6da4bded9f None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:23.134662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b981c3cc-383a-4f22-bd7d-b6405148a439 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:23.139746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-b981c3cc-383a-4f22-bd7d-b6405148a439 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:35:23.139746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:23.139746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:23.139746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:23.139746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:23.139746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:23.139746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:23.139746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:23.139746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:23.139746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:23.139746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:23.139746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:23.139746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:23.139746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:23.139746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:23.139746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:23.140697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:23.140697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:23.140697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:23.140697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:23.140697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:23.140697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:23.140697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:23.140697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:23.140697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:23.140697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:23.140697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:23.140697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:23.140697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:23.140697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:23.140697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:23.140697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:23.141532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:23.141532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:23.141532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:23.141532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:23.141532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:23.141532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:23.141532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:23.141532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:23.141532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:23.141532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:23.141532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:35:23.141532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:23.141532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 158/321] 10.222.0.36 () {56 vars in 1114 bytes} [Sun Oct 30 00:35:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:23.142229 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981c3cc-383a-4f22-bd7d-b6405148a439 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:23.186712 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b10a81-0097-42af-8a6b-2f837caa11d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:23.187626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b10a81-0097-42af-8a6b-2f837caa11d3 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:23.187919 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b10a81-0097-42af-8a6b-2f837caa11d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:23.188229 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b10a81-0097-42af-8a6b-2f837caa11d3 None admin] PATH_INFO: `/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:23.191691 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b10a81-0097-42af-8a6b-2f837caa11d3 None admin] RBAC: Authorizing `identity:get_project(project_id=b9c6b20f7b7a4967ada2d4c9d25dd94a)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:23.193284 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b10a81-0097-42af-8a6b-2f837caa11d3 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:23.195141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 164/322] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:35:22 2022] GET /identity/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:23.195772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b10a81-0097-42af-8a6b-2f837caa11d3 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:23.304029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981c3cc-383a-4f22-bd7d-b6405148a439 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:23.309627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:23.331586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b10a81-0097-42af-8a6b-2f837caa11d3 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:23.340526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98896e5f-62e8-44ed-b35b-75dfad0cf711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:23.341140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98896e5f-62e8-44ed-b35b-75dfad0cf711 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:23.341428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98896e5f-62e8-44ed-b35b-75dfad0cf711 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:23.341699 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98896e5f-62e8-44ed-b35b-75dfad0cf711 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:23.342427 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98896e5f-62e8-44ed-b35b-75dfad0cf711 None admin] RBAC: Adding query filter params (group.id=d04550585c1b41f493f4a776f88ce662, role.id=af3529d63bae4650be7b8dc67b3ed527, scope.project.id=72d02777c6694301959a12c02452adf3) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:23.342740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98896e5f-62e8-44ed-b35b-75dfad0cf711 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:23.344451 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98896e5f-62e8-44ed-b35b-75dfad0cf711 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:23.350408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 159/323] 10.222.0.36 () {56 vars in 1376 bytes} [Sun Oct 30 00:35:23 2022] GET /identity/v3/role_assignments?group.id=d04550585c1b41f493f4a776f88ce662&scope.project.id=72d02777c6694301959a12c02452adf3&role.id=af3529d63bae4650be7b8dc67b3ed527 => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:23.351223 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98896e5f-62e8-44ed-b35b-75dfad0cf711 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:23.376010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2687f28-b467-4b3d-92f8-80c8542cb359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:23.376664 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2687f28-b467-4b3d-92f8-80c8542cb359 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:23.376944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2687f28-b467-4b3d-92f8-80c8542cb359 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:23.377217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2687f28-b467-4b3d-92f8-80c8542cb359 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:23.377905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2687f28-b467-4b3d-92f8-80c8542cb359 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:23.378204 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2687f28-b467-4b3d-92f8-80c8542cb359 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:23.379645 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2687f28-b467-4b3d-92f8-80c8542cb359 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:23.389128 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:23.395607 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 165/324] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:23.395607 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2687f28-b467-4b3d-92f8-80c8542cb359 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:23.483197 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98896e5f-62e8-44ed-b35b-75dfad0cf711 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:23.510934 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2687f28-b467-4b3d-92f8-80c8542cb359 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:23.528610 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7561da-4c89-4bb2-9580-a6e4dc704fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:23.529277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7561da-4c89-4bb2-9580-a6e4dc704fb3 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:23.529701 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7561da-4c89-4bb2-9580-a6e4dc704fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:23.529981 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7561da-4c89-4bb2-9580-a6e4dc704fb3 None admin] PATH_INFO: `/v3/groups/d04550585c1b41f493f4a776f88ce662` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:23.531759 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7561da-4c89-4bb2-9580-a6e4dc704fb3 None admin] RBAC: Authorizing `identity:get_group(group_id=d04550585c1b41f493f4a776f88ce662)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:23.533363 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7561da-4c89-4bb2-9580-a6e4dc704fb3 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:23.535187 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 160/325] 10.222.0.36 () {56 vars in 1180 bytes} [Sun Oct 30 00:35:23 2022] GET /identity/v3/groups/d04550585c1b41f493f4a776f88ce662 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:23.536897 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7561da-4c89-4bb2-9580-a6e4dc704fb3 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:23.562660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cacd29-23b9-4c0d-a763-f911ce1d1330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:23.563892 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cacd29-23b9-4c0d-a763-f911ce1d1330 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:23.563892 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cacd29-23b9-4c0d-a763-f911ce1d1330 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:23.564112 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cacd29-23b9-4c0d-a763-f911ce1d1330 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:23.573899 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:23.576919 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cacd29-23b9-4c0d-a763-f911ce1d1330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:23.583243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-08cacd29-23b9-4c0d-a763-f911ce1d1330 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:35:23.583243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:23.583243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:23.583243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:23.583243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:23.583243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:23.583243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:23.583243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:23.583243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:23.583243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:23.583243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:23.583243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:23.583243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:23.583243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:23.583243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:23.583243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:23.584351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:23.584351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:23.584351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:23.584351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:23.584351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:23.584351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:23.584351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:23.584351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:23.584351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:23.584351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:23.584351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:23.584351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:23.584351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:23.584351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:23.584351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:23.584351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:23.585309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:23.585309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:23.585309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:23.585309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:23.585309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:23.585309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:23.585309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:23.585309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:23.585309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:23.585309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:23.585309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:35:23.585309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:23.589088 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 166/326] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:35:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:23.591381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cacd29-23b9-4c0d-a763-f911ce1d1330 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:23.659869 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:23.665903 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7561da-4c89-4bb2-9580-a6e4dc704fb3 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:23.709125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468a041-a1d2-40f4-bfdf-d72f691e242a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:23.709756 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468a041-a1d2-40f4-bfdf-d72f691e242a None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:23.710040 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468a041-a1d2-40f4-bfdf-d72f691e242a None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:23.710315 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468a041-a1d2-40f4-bfdf-d72f691e242a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:23.710906 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cacd29-23b9-4c0d-a763-f911ce1d1330 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:23.755928 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcd063-cb7a-4188-84d3-ceb42621b2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:23.757308 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcd063-cb7a-4188-84d3-ceb42621b2bc None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:23.757308 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcd063-cb7a-4188-84d3-ceb42621b2bc None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:23.757308 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcd063-cb7a-4188-84d3-ceb42621b2bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:23.757813 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bcd063-cb7a-4188-84d3-ceb42621b2bc None admin] RBAC: Adding query filter params (group.id=d04550585c1b41f493f4a776f88ce662, role.id=2e0593ed85a14197a0eaf37135a01787, scope.project.id=b9c6b20f7b7a4967ada2d4c9d25dd94a) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:23.758071 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bcd063-cb7a-4188-84d3-ceb42621b2bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:23.760639 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bcd063-cb7a-4188-84d3-ceb42621b2bc None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:23.767664 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 167/327] 10.222.0.36 () {56 vars in 1376 bytes} [Sun Oct 30 00:35:23 2022] GET /identity/v3/role_assignments?group.id=d04550585c1b41f493f4a776f88ce662&scope.project.id=b9c6b20f7b7a4967ada2d4c9d25dd94a&role.id=2e0593ed85a14197a0eaf37135a01787 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:23.768417 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcd063-cb7a-4188-84d3-ceb42621b2bc None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:23.798624 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468a041-a1d2-40f4-bfdf-d72f691e242a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:23.800744 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468a041-a1d2-40f4-bfdf-d72f691e242a None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:23.838711 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:23.893707 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcd063-cb7a-4188-84d3-ceb42621b2bc None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:23.921945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:23.931658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 161/328] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:35:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:35:23.932938 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468a041-a1d2-40f4-bfdf-d72f691e242a None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:23.936883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2dc59e-5489-4fea-9538-a0bbaf27fbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:23.936883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2dc59e-5489-4fea-9538-a0bbaf27fbf3 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:23.937144 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2dc59e-5489-4fea-9538-a0bbaf27fbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:23.937230 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2dc59e-5489-4fea-9538-a0bbaf27fbf3 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:23.937919 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2dc59e-5489-4fea-9538-a0bbaf27fbf3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:23.938227 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2dc59e-5489-4fea-9538-a0bbaf27fbf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:23.939650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2dc59e-5489-4fea-9538-a0bbaf27fbf3 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:23.951507 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 168/329] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:23.957410 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2dc59e-5489-4fea-9538-a0bbaf27fbf3 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:24.058127 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468a041-a1d2-40f4-bfdf-d72f691e242a None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:24.077371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2dc59e-5489-4fea-9538-a0bbaf27fbf3 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:24.104726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f0659-27be-420b-902d-a7fab1829313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:24.105924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f0659-27be-420b-902d-a7fab1829313 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:24.106190 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f0659-27be-420b-902d-a7fab1829313 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:24.106377 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f0659-27be-420b-902d-a7fab1829313 None admin] PATH_INFO: `/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:24.107451 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:24.107557 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f0659-27be-420b-902d-a7fab1829313 None admin] RBAC: Authorizing `identity:get_project(project_id=b9c6b20f7b7a4967ada2d4c9d25dd94a)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:24.109054 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f0659-27be-420b-902d-a7fab1829313 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:24.110830 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 162/330] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:35:23 2022] GET /identity/v3/projects/b9c6b20f7b7a4967ada2d4c9d25dd94a => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:24.117123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f0659-27be-420b-902d-a7fab1829313 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:24.118097 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f6653-9680-4d7a-8f6d-1069a6ef8121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:24.118869 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f6653-9680-4d7a-8f6d-1069a6ef8121 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:24.119157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f6653-9680-4d7a-8f6d-1069a6ef8121 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:24.119434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f6653-9680-4d7a-8f6d-1069a6ef8121 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:24.195261 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:24.201080 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f6653-9680-4d7a-8f6d-1069a6ef8121 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:24.202561 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f6653-9680-4d7a-8f6d-1069a6ef8121 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:24.235335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f0659-27be-420b-902d-a7fab1829313 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:24.269230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2812078-c0f9-4d81-8eb9-a887bb5d11b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:24.269868 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2812078-c0f9-4d81-8eb9-a887bb5d11b9 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:24.270146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2812078-c0f9-4d81-8eb9-a887bb5d11b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:24.270419 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2812078-c0f9-4d81-8eb9-a887bb5d11b9 None admin] PATH_INFO: `/v3/groups/d04550585c1b41f493f4a776f88ce662` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:24.271520 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2812078-c0f9-4d81-8eb9-a887bb5d11b9 None admin] RBAC: Authorizing `identity:get_group(group_id=d04550585c1b41f493f4a776f88ce662)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:24.272985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2812078-c0f9-4d81-8eb9-a887bb5d11b9 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:24.275251 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 163/331] 10.222.0.36 () {56 vars in 1180 bytes} [Sun Oct 30 00:35:24 2022] GET /identity/v3/groups/d04550585c1b41f493f4a776f88ce662 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:24.280905 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2812078-c0f9-4d81-8eb9-a887bb5d11b9 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:24.305632 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 169/332] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:35:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 30 00:35:24.311136 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f6653-9680-4d7a-8f6d-1069a6ef8121 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:24.336999 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:24.380947 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2812078-c0f9-4d81-8eb9-a887bb5d11b9 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:24.412317 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f6653-9680-4d7a-8f6d-1069a6ef8121 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:24.415339 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f9140-d6ed-4890-8d27-6f3a1c4008ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:24.415953 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f9140-d6ed-4890-8d27-6f3a1c4008ff None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:24.416265 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f9140-d6ed-4890-8d27-6f3a1c4008ff None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:24.416542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f9140-d6ed-4890-8d27-6f3a1c4008ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:24.417267 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f9140-d6ed-4890-8d27-6f3a1c4008ff None admin] RBAC: Adding query filter params (group.id=d04550585c1b41f493f4a776f88ce662, role.id=af3529d63bae4650be7b8dc67b3ed527, scope.project.id=b9c6b20f7b7a4967ada2d4c9d25dd94a) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:24.417576 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f9140-d6ed-4890-8d27-6f3a1c4008ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:24.419009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f9140-d6ed-4890-8d27-6f3a1c4008ff None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:24.422051 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:24.432287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 164/333] 10.222.0.36 () {56 vars in 1376 bytes} [Sun Oct 30 00:35:24 2022] GET /identity/v3/role_assignments?group.id=d04550585c1b41f493f4a776f88ce662&scope.project.id=b9c6b20f7b7a4967ada2d4c9d25dd94a&role.id=af3529d63bae4650be7b8dc67b3ed527 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:24.450414 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6a5c8-4424-40e3-b7af-0064654fed42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:24.451070 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6a5c8-4424-40e3-b7af-0064654fed42 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:24.451357 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6a5c8-4424-40e3-b7af-0064654fed42 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:24.451631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6a5c8-4424-40e3-b7af-0064654fed42 None admin] PATH_INFO: `/v3/projects/3c2609b991ac4a86bca190e144b13290` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:24.452867 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6a5c8-4424-40e3-b7af-0064654fed42 None admin] RBAC: Authorizing `identity:get_project(project_id=3c2609b991ac4a86bca190e144b13290)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:24.454367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6a5c8-4424-40e3-b7af-0064654fed42 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:24.456328 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 170/334] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:35:24 2022] GET /identity/v3/projects/3c2609b991ac4a86bca190e144b13290 => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:24.463015 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f9140-d6ed-4890-8d27-6f3a1c4008ff None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:24.572043 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f9140-d6ed-4890-8d27-6f3a1c4008ff None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:24.610483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e904d26-1bf7-4bc9-a9d4-c6e9e773efb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:24.611127 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e904d26-1bf7-4bc9-a9d4-c6e9e773efb0 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:24.611470 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e904d26-1bf7-4bc9-a9d4-c6e9e773efb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:24.611758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e904d26-1bf7-4bc9-a9d4-c6e9e773efb0 None admin] PATH_INFO: `/v3/groups/38243fea584d42b7b0ac1d5f5cb63238` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:24.613403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e904d26-1bf7-4bc9-a9d4-c6e9e773efb0 None admin] RBAC: Authorizing `identity:get_group(group_id=38243fea584d42b7b0ac1d5f5cb63238)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:24.614862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e904d26-1bf7-4bc9-a9d4-c6e9e773efb0 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:24.616484 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 165/335] 10.222.0.36 () {56 vars in 1180 bytes} [Sun Oct 30 00:35:24 2022] GET /identity/v3/groups/38243fea584d42b7b0ac1d5f5cb63238 => generated 229 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:24.622654 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6a5c8-4424-40e3-b7af-0064654fed42 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:24.729469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:24.737341 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6a5c8-4424-40e3-b7af-0064654fed42 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:24.773342 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f64fa4-9eca-45e8-97b7-a8fdf6d5b23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:24.774616 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f64fa4-9eca-45e8-97b7-a8fdf6d5b23f None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:24.775060 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f64fa4-9eca-45e8-97b7-a8fdf6d5b23f None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:24.775353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f64fa4-9eca-45e8-97b7-a8fdf6d5b23f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:24.776122 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f64fa4-9eca-45e8-97b7-a8fdf6d5b23f None admin] RBAC: Adding query filter params (group.id=38243fea584d42b7b0ac1d5f5cb63238, role.id=980241d45fdb435d9f32c434057c71e4, scope.project.id=3c2609b991ac4a86bca190e144b13290) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:24.776436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f64fa4-9eca-45e8-97b7-a8fdf6d5b23f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:24.777887 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f64fa4-9eca-45e8-97b7-a8fdf6d5b23f None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:24.784221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 171/336] 10.222.0.36 () {56 vars in 1376 bytes} [Sun Oct 30 00:35:24 2022] GET /identity/v3/role_assignments?group.id=38243fea584d42b7b0ac1d5f5cb63238&scope.project.id=3c2609b991ac4a86bca190e144b13290&role.id=980241d45fdb435d9f32c434057c71e4 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:27.582081 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07a51b-366b-401e-8c35-b997bd5fc6dc None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:27.582628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07a51b-366b-401e-8c35-b997bd5fc6dc None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:27.582628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07a51b-366b-401e-8c35-b997bd5fc6dc None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:27.584255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 166/337] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:27.590285 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26c2b3-7c4b-4364-9ccc-e727f8270249 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:27.590872 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21433a1d-e24e-49e0-9172-421ba03c18d7 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:27.590955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26c2b3-7c4b-4364-9ccc-e727f8270249 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:27.591035 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21433a1d-e24e-49e0-9172-421ba03c18d7 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:27.591111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26c2b3-7c4b-4364-9ccc-e727f8270249 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:27.591179 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21433a1d-e24e-49e0-9172-421ba03c18d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:27.592240 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 172/338] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:27.614854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-21433a1d-e24e-49e0-9172-421ba03c18d7 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:27.616069 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-21433a1d-e24e-49e0-9172-421ba03c18d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:27.620892 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6172f5-7b92-4923-9885-2c0d8c173466 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:27.621175 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6172f5-7b92-4923-9885-2c0d8c173466 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:27.621452 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6172f5-7b92-4923-9885-2c0d8c173466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:27.649310 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-0d6172f5-7b92-4923-9885-2c0d8c173466 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:27.651572 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-0d6172f5-7b92-4923-9885-2c0d8c173466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:27.721729 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:27.735775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 167/339] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:35:27.768599 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 173/340] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:27.878259 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45edfa-c059-4742-a752-5c520c0e6096 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:27.879075 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45edfa-c059-4742-a752-5c520c0e6096 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:27.879759 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45edfa-c059-4742-a752-5c520c0e6096 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:27.882362 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 168/341] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:27.889169 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbdb40-f800-4d68-ac8e-7e5bd57dd7da None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:27.890071 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbdb40-f800-4d68-ac8e-7e5bd57dd7da None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:27.890749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbdb40-f800-4d68-ac8e-7e5bd57dd7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:27.922015 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-91fbdb40-f800-4d68-ac8e-7e5bd57dd7da None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:27.923235 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-91fbdb40-f800-4d68-ac8e-7e5bd57dd7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:28.017671 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 174/342] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:28.100049 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45edfa-c059-4742-a752-5c520c0e6096 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:28.149600 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbdb40-f800-4d68-ac8e-7e5bd57dd7da None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:28.184454 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:28.277246 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45edfa-c059-4742-a752-5c520c0e6096 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:28.334150 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbdb40-f800-4d68-ac8e-7e5bd57dd7da None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:28.348555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4f567-bfef-446f-b281-5f7e414dc71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:28.349679 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4f567-bfef-446f-b281-5f7e414dc71f None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:28.350176 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4f567-bfef-446f-b281-5f7e414dc71f None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:28.350663 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4f567-bfef-446f-b281-5f7e414dc71f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:28.362564 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4f567-bfef-446f-b281-5f7e414dc71f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:28.364456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4f567-bfef-446f-b281-5f7e414dc71f None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:28.380564 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808de578-25b1-4d73-9a23-9d811067953e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:28.381441 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808de578-25b1-4d73-9a23-9d811067953e None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:28.381868 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808de578-25b1-4d73-9a23-9d811067953e None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:28.382098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-e8d4f567-bfef-446f-b281-5f7e414dc71f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:28.382098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:28.382098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:28.382098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:28.382098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:28.382098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:28.382098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:28.382098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:28.382098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:28.382098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:28.382098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:28.382098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:28.382098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:28.382098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:28.382098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:28.382098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:28.383021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:28.383021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:28.383021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:28.383021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:28.383021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:28.383021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:28.383021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:28.383021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:28.383021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:28.383021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:28.383021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:28.383021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:28.383021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:28.383021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:28.383021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:28.383021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:28.383021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:28.384797 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:28.384797 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:28.384797 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:28.384797 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 169/343] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:28.385467 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808de578-25b1-4d73-9a23-9d811067953e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:28.388880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4f567-bfef-446f-b281-5f7e414dc71f None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:28.397300 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808de578-25b1-4d73-9a23-9d811067953e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:28.399207 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808de578-25b1-4d73-9a23-9d811067953e None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:28.419119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-808de578-25b1-4d73-9a23-9d811067953e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:28.419119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:28.419119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:28.419119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:28.419119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:28.419119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:28.419119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:28.419119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:28.419119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:28.419119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:28.419119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:28.419119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:28.419119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:28.419119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:28.419119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:28.419119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:28.420024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:28.420024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:28.420024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:28.420024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:28.420024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:28.420024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:28.420024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:28.420024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:28.420024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:28.420024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:28.420024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:28.420024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:28.420024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:28.420024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:28.420024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:28.420024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:28.420024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:28.420968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:28.420968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:28.420968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:28.422922 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 175/344] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:28.424066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808de578-25b1-4d73-9a23-9d811067953e None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:28.445021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:28.497553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:28.505239 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4f567-bfef-446f-b281-5f7e414dc71f None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:28.531025 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808de578-25b1-4d73-9a23-9d811067953e None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:28.547405 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809b87a-b32a-47a9-a19e-c009e6987b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:28.548041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809b87a-b32a-47a9-a19e-c009e6987b60 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:28.548360 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809b87a-b32a-47a9-a19e-c009e6987b60 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:28.548633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809b87a-b32a-47a9-a19e-c009e6987b60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:28.561531 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809b87a-b32a-47a9-a19e-c009e6987b60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:28.563233 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809b87a-b32a-47a9-a19e-c009e6987b60 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:28.567443 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53f525-bf45-4977-85ac-07858d61033d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:28.568055 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53f525-bf45-4977-85ac-07858d61033d None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:28.568358 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53f525-bf45-4977-85ac-07858d61033d None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:28.568628 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53f525-bf45-4977-85ac-07858d61033d None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:28.569274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53f525-bf45-4977-85ac-07858d61033d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:28.569565 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53f525-bf45-4977-85ac-07858d61033d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:28.570934 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53f525-bf45-4977-85ac-07858d61033d None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:28.579080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-b809b87a-b32a-47a9-a19e-c009e6987b60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:28.579080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:28.579080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:28.579080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:28.579080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:28.579080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:28.579080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:28.579080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:28.579080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:28.579080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:28.579080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:28.579080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:28.579080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:28.579080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:28.579080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:28.579080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:28.580041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:28.580041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:28.580041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:28.580041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:28.580041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:28.580041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:28.580041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:28.580041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:28.580041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:28.580041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:28.580041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:28.580041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:28.580041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:28.580041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:28.580041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:28.580041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:28.580041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:28.586635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:28.586635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:28.586635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:28.586635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 170/345] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:28.586635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bb0e1-81f0-4dfc-a6cb-5e8d5231649a None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:28.586635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bb0e1-81f0-4dfc-a6cb-5e8d5231649a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:28.586635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bb0e1-81f0-4dfc-a6cb-5e8d5231649a None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:28.586635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 171/346] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:28.586635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7bb0e1-81f0-4dfc-a6cb-5e8d5231649a None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:28.596507 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 176/347] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:28.597306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53f525-bf45-4977-85ac-07858d61033d None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:28.699514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7bb0e1-81f0-4dfc-a6cb-5e8d5231649a None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:28.701306 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:28.704734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53f525-bf45-4977-85ac-07858d61033d None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:28.736102 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:28.738289 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e56756-5fbd-48b2-9fa7-efb4a3c019ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:28.738904 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e56756-5fbd-48b2-9fa7-efb4a3c019ce None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:28.739183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e56756-5fbd-48b2-9fa7-efb4a3c019ce None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:28.739443 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e56756-5fbd-48b2-9fa7-efb4a3c019ce None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:28.740112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e56756-5fbd-48b2-9fa7-efb4a3c019ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:28.740414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e56756-5fbd-48b2-9fa7-efb4a3c019ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:28.741796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e56756-5fbd-48b2-9fa7-efb4a3c019ce None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:28.743704 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c5bd6-f0a1-4fee-96fa-26eec5b1f7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:28.744622 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c5bd6-f0a1-4fee-96fa-26eec5b1f7d6 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:28.745407 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c5bd6-f0a1-4fee-96fa-26eec5b1f7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:28.745740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c5bd6-f0a1-4fee-96fa-26eec5b1f7d6 None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:28.746344 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c5bd6-f0a1-4fee-96fa-26eec5b1f7d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:28.746647 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c5bd6-f0a1-4fee-96fa-26eec5b1f7d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:28.748058 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c5bd6-f0a1-4fee-96fa-26eec5b1f7d6 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:28.755331 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 172/348] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:28.757550 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b9010-ce7d-4fa0-a9e1-5cdb1b89e31a None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:28.758054 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b9010-ce7d-4fa0-a9e1-5cdb1b89e31a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:28.759413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b9010-ce7d-4fa0-a9e1-5cdb1b89e31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:28.763105 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 177/349] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:28.763998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c5bd6-f0a1-4fee-96fa-26eec5b1f7d6 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:28.781073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-2f3b9010-ce7d-4fa0-a9e1-5cdb1b89e31a None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:28.782374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-2f3b9010-ce7d-4fa0-a9e1-5cdb1b89e31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:28.868392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 173/350] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:28.868392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b9010-ce7d-4fa0-a9e1-5cdb1b89e31a None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:28.896731 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c5bd6-f0a1-4fee-96fa-26eec5b1f7d6 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:28.936791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c845419-8d5b-4a7a-b28e-2c9d2b193e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:28.937410 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c845419-8d5b-4a7a-b28e-2c9d2b193e93 None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:28.937689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c845419-8d5b-4a7a-b28e-2c9d2b193e93 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:28.937962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c845419-8d5b-4a7a-b28e-2c9d2b193e93 None admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:28.938612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c845419-8d5b-4a7a-b28e-2c9d2b193e93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:28.942182 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c845419-8d5b-4a7a-b28e-2c9d2b193e93 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:28.976522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 178/351] 10.222.0.36 () {60 vars in 1132 bytes} [Sun Oct 30 00:35:28 2022] POST /identity/v3/users => generated 248 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:35:28.976522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c845419-8d5b-4a7a-b28e-2c9d2b193e93 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:28.993189 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:29.012106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b9010-ce7d-4fa0-a9e1-5cdb1b89e31a None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:29.077842 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:29.084205 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75661d-0676-43f2-9939-a97f1a894504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:29.084830 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75661d-0676-43f2-9939-a97f1a894504 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:29.085106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75661d-0676-43f2-9939-a97f1a894504 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:29.085425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75661d-0676-43f2-9939-a97f1a894504 None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:29.086079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75661d-0676-43f2-9939-a97f1a894504 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:29.088075 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75661d-0676-43f2-9939-a97f1a894504 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:29.130094 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 174/352] 10.222.0.36 () {60 vars in 1132 bytes} [Sun Oct 30 00:35:28 2022] POST /identity/v3/users => generated 246 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:35:29.170372 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c845419-8d5b-4a7a-b28e-2c9d2b193e93 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:29.176121 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75661d-0676-43f2-9939-a97f1a894504 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:29.241089 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57a9db-267a-42be-907c-e5c1e746f3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:29.241656 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57a9db-267a-42be-907c-e5c1e746f3d6 None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:29.241940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57a9db-267a-42be-907c-e5c1e746f3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:29.242217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57a9db-267a-42be-907c-e5c1e746f3d6 None admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:29.242928 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57a9db-267a-42be-907c-e5c1e746f3d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:29.244597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57a9db-267a-42be-907c-e5c1e746f3d6 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:29.277845 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 179/353] 10.222.0.36 () {60 vars in 1132 bytes} [Sun Oct 30 00:35:28 2022] POST /identity/v3/users => generated 248 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:35:29.309771 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e110f-4fd3-44af-9415-aa977480585b None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:29.310127 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e110f-4fd3-44af-9415-aa977480585b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:29.310334 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e110f-4fd3-44af-9415-aa977480585b None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:29.311983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 180/354] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:35:29.314268 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75661d-0676-43f2-9939-a97f1a894504 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:29.319209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7d189-7b5f-4dc8-baf9-4b5ca1108f88 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:29.319522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7d189-7b5f-4dc8-baf9-4b5ca1108f88 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:29.319770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7d189-7b5f-4dc8-baf9-4b5ca1108f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:29.344929 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-d1e7d189-7b5f-4dc8-baf9-4b5ca1108f88 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:29.346258 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d1e7d189-7b5f-4dc8-baf9-4b5ca1108f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:29.370759 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d4248-e7f8-4ce7-a9c4-13d99078eb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:29.371949 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d4248-e7f8-4ce7-a9c4-13d99078eb19 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:29.372467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d4248-e7f8-4ce7-a9c4-13d99078eb19 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:29.373088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d4248-e7f8-4ce7-a9c4-13d99078eb19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:29.389304 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d4248-e7f8-4ce7-a9c4-13d99078eb19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:29.390821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d4248-e7f8-4ce7-a9c4-13d99078eb19 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:29.407312 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:29.420012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-5d4d4248-e7f8-4ce7-a9c4-13d99078eb19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:29.420012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:29.420012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:29.420012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:29.420012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:29.420012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:29.420012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:29.420012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:29.420012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:29.420012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:29.420012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:29.420012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:29.420012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:29.420012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:29.420012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:29.420012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:29.421080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:29.421080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:29.421080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:29.421080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:29.421080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:29.421080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:29.421080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:29.421080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:29.421080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:29.421080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:29.421080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:29.421080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:29.421080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:29.421080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:29.421080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:29.421080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:29.421080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:29.422036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:29.422036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:29.422036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:29.424019 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 175/355] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:35:29.432574 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d4248-e7f8-4ce7-a9c4-13d99078eb19 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:29.449900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 181/356] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:35:29.563320 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d4248-e7f8-4ce7-a9c4-13d99078eb19 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:29.603797 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ac23d-a87e-494a-8bfc-95c42f88953f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:29.604468 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ac23d-a87e-494a-8bfc-95c42f88953f None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:29.604756 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ac23d-a87e-494a-8bfc-95c42f88953f None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:29.605034 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ac23d-a87e-494a-8bfc-95c42f88953f None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:29.605688 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ac23d-a87e-494a-8bfc-95c42f88953f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:29.605987 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ac23d-a87e-494a-8bfc-95c42f88953f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:29.607423 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ac23d-a87e-494a-8bfc-95c42f88953f None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:29.625464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 176/357] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:29.631433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7d189-7b5f-4dc8-baf9-4b5ca1108f88 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:29.675898 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:29.769305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7d189-7b5f-4dc8-baf9-4b5ca1108f88 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:29.800067 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ac23d-a87e-494a-8bfc-95c42f88953f None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:29.823472 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d30670-3f64-44d8-b2b9-bd675d328e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:29.824456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d30670-3f64-44d8-b2b9-bd675d328e80 None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:29.824965 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d30670-3f64-44d8-b2b9-bd675d328e80 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:29.825457 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d30670-3f64-44d8-b2b9-bd675d328e80 None admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:29.826330 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d30670-3f64-44d8-b2b9-bd675d328e80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:29.828006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d30670-3f64-44d8-b2b9-bd675d328e80 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:29.856906 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 182/358] 10.222.0.36 () {60 vars in 1132 bytes} [Sun Oct 30 00:35:29 2022] POST /identity/v3/users => generated 249 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:35:29.903850 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:29.968431 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ac23d-a87e-494a-8bfc-95c42f88953f None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:30.016349 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41c4b6-fe5f-4abd-8e3d-8a3de79a6107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:30.016968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41c4b6-fe5f-4abd-8e3d-8a3de79a6107 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:30.017250 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41c4b6-fe5f-4abd-8e3d-8a3de79a6107 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:30.017526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41c4b6-fe5f-4abd-8e3d-8a3de79a6107 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:30.018210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41c4b6-fe5f-4abd-8e3d-8a3de79a6107 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:30.019584 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41c4b6-fe5f-4abd-8e3d-8a3de79a6107 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:30.033848 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.sql.core [None req-aa41c4b6-fe5f-4abd-8e3d-8a3de79a6107 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 30 00:35:30.033848 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 30 00:35:30.033848 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [parameters: {'id': '058417948025400b9b248b2c402fddab', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 30 00:35:30.033848 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88201) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 30 00:35:30.041140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-aa41c4b6-fe5f-4abd-8e3d-8a3de79a6107 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 30 00:35:30.041140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:30.041140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:35:30.041140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:35:30.041140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:35:30.041140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:35:30.041140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:35:30.041140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:35:30.041140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:35:30.041140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:35:30.041140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:35:30.041140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:35:30.041140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:35:30.041140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application result.read() Oct 30 00:35:30.041140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:35:30.042138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:35:30.042138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:35:30.042138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:35:30.042138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:35:30.042138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:35:30.042138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:35:30.042138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:35:30.042138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 30 00:35:30.042138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:30.042138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 30 00:35:30.042138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:30.042138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:30.042138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 30 00:35:30.042138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 30 00:35:30.042138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 30 00:35:30.042138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return ref.to_dict() Oct 30 00:35:30.042138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:35:30.042138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:35:30.043079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 30 00:35:30.043079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application yield resource Oct 30 00:35:30.043079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:35:30.043079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:35:30.043079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 30 00:35:30.043079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.session.rollback() Oct 30 00:35:30.043079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 30 00:35:30.043079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.force_reraise() Oct 30 00:35:30.043079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 30 00:35:30.043079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise self.value Oct 30 00:35:30.043079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 30 00:35:30.043079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 30 00:35:30.043079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 30 00:35:30.043079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application session.commit() Oct 30 00:35:30.043079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 30 00:35:30.043079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 30 00:35:30.043079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 30 00:35:30.043945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._prepare_impl() Oct 30 00:35:30.043945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 30 00:35:30.043945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.session.flush() Oct 30 00:35:30.043945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 30 00:35:30.043945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._flush(objects) Oct 30 00:35:30.043945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 30 00:35:30.043945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 30 00:35:30.043945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 30 00:35:30.043945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application compat.raise_( Oct 30 00:35:30.043945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:35:30.043945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception Oct 30 00:35:30.043945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 30 00:35:30.043945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application flush_context.execute() Oct 30 00:35:30.043945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 30 00:35:30.043945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rec.execute(self) Oct 30 00:35:30.043945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 30 00:35:30.043945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 30 00:35:30.044858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 30 00:35:30.044858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 30 00:35:30.044858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 30 00:35:30.044858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 30 00:35:30.044858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 30 00:35:30.044858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 30 00:35:30.044858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 30 00:35:30.044858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 30 00:35:30.044858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 30 00:35:30.044858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 30 00:35:30.044858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 30 00:35:30.044858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 30 00:35:30.044858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 30 00:35:30.044858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 30 00:35:30.044858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:35:30.044858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception Oct 30 00:35:30.045702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:35:30.045702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:35:30.045702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:35:30.045702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:35:30.045702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:35:30.045702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:35:30.045702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:35:30.045702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:35:30.045702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:35:30.045702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:35:30.045702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:35:30.045702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application result.read() Oct 30 00:35:30.045702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:35:30.045702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:35:30.045702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:35:30.045702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:35:30.046506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:35:30.046506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:35:30.046506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:35:30.046506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:35:30.046506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 30 00:35:30.046506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 30 00:35:30.046506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [parameters: {'id': '058417948025400b9b248b2c402fddab', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 30 00:35:30.046506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 30 00:35:30.046506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:30.046506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:35:30.046506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:30.046506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:30.046506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:30.046506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:30.046506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:30.046506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:30.049023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:30.049023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:30.049023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:30.049023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:30.049023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:30.049023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:30.049023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 30 00:35:30.049023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 30 00:35:30.049023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:30.049023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:30.049023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 30 00:35:30.049023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 30 00:35:30.049023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 30 00:35:30.049023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 30 00:35:30.049023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 30 00:35:30.049023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:30.051032 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 177/359] 10.222.0.36 () {60 vars in 1131 bytes} [Sun Oct 30 00:35:29 2022] POST /identity/v3/roles => generated 144 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:30.051177 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d30670-3f64-44d8-b2b9-bd675d328e80 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:30.160030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:30.169607 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d30670-3f64-44d8-b2b9-bd675d328e80 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:30.210924 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b202df-084b-436b-90df-7c48deaab576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:30.211601 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b202df-084b-436b-90df-7c48deaab576 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:30.212666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b202df-084b-436b-90df-7c48deaab576 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:30.212666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b202df-084b-436b-90df-7c48deaab576 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:30.224559 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b202df-084b-436b-90df-7c48deaab576 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:30.231311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-96b202df-084b-436b-90df-7c48deaab576 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:35:30.231311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:30.231311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:30.231311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:30.231311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:30.231311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:30.231311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:30.231311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:30.231311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:30.231311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:30.231311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:30.231311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:30.231311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:30.231311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:30.231311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:30.231311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:30.232462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:30.232462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:30.232462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:30.232462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:30.232462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:30.232462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:30.232462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:30.232462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:30.232462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:30.232462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:30.232462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:30.232462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:30.232462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:30.232462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:30.232462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:30.232462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:30.233848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:30.233848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:30.233848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:30.233848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:30.233848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:30.233848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:30.233848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:30.233848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:30.233848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:30.233848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:30.233848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 30 00:35:30.233848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:30.233848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 183/360] 10.222.0.36 () {56 vars in 1114 bytes} [Sun Oct 30 00:35:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:30.244172 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41c4b6-fe5f-4abd-8e3d-8a3de79a6107 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:30.364503 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41c4b6-fe5f-4abd-8e3d-8a3de79a6107 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:30.396876 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:30.413467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e51ce5-7800-4f60-a288-2e33e232b723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:30.414562 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e51ce5-7800-4f60-a288-2e33e232b723 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:30.415267 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e51ce5-7800-4f60-a288-2e33e232b723 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:30.417211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e51ce5-7800-4f60-a288-2e33e232b723 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:30.418762 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e51ce5-7800-4f60-a288-2e33e232b723 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:30.419572 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e51ce5-7800-4f60-a288-2e33e232b723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:30.421297 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e51ce5-7800-4f60-a288-2e33e232b723 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:30.436694 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 178/361] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:31.759854 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa81d2-6c7e-48a0-903e-95a77d069f1d None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:31.760392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa81d2-6c7e-48a0-903e-95a77d069f1d None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:31.760467 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa81d2-6c7e-48a0-903e-95a77d069f1d None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:31.762129 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 184/362] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:31.767691 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecfb13-c64e-4115-8903-e383622fca9f None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:31.768175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecfb13-c64e-4115-8903-e383622fca9f None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:31.768357 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecfb13-c64e-4115-8903-e383622fca9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:31.793096 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-34ecfb13-c64e-4115-8903-e383622fca9f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:31.794305 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-34ecfb13-c64e-4115-8903-e383622fca9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:31.903242 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 179/363] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:32.016219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e340b7-cb98-4950-883a-fb7b8a198559 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:32.016510 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e340b7-cb98-4950-883a-fb7b8a198559 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:32.016685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e340b7-cb98-4950-883a-fb7b8a198559 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:32.018263 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 185/364] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:32.023683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957c269-0462-4b4a-838f-fd372d4c93d1 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:32.023973 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957c269-0462-4b4a-838f-fd372d4c93d1 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:32.024300 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957c269-0462-4b4a-838f-fd372d4c93d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:32.053091 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-4957c269-0462-4b4a-838f-fd372d4c93d1 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:32.054293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4957c269-0462-4b4a-838f-fd372d4c93d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:32.154984 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 180/365] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:32.225778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e340b7-cb98-4950-883a-fb7b8a198559 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:32.330905 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef2c72-0cac-44b5-978e-c60d6343abc8 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:32.331196 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef2c72-0cac-44b5-978e-c60d6343abc8 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:32.331388 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef2c72-0cac-44b5-978e-c60d6343abc8 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:32.332790 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 181/366] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:32.337948 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d710e-7eaf-432b-ad08-a94aed3c844f None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:32.338246 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d710e-7eaf-432b-ad08-a94aed3c844f None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:32.338523 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d710e-7eaf-432b-ad08-a94aed3c844f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:32.346836 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e340b7-cb98-4950-883a-fb7b8a198559 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:32.373229 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-cc9d710e-7eaf-432b-ad08-a94aed3c844f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:32.374470 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-cc9d710e-7eaf-432b-ad08-a94aed3c844f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:32.386527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4f34c-9c9f-4556-8275-ae00c04985e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:32.387283 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4f34c-9c9f-4556-8275-ae00c04985e4 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:32.387709 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4f34c-9c9f-4556-8275-ae00c04985e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:32.388038 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4f34c-9c9f-4556-8275-ae00c04985e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:32.399598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c4f34c-9c9f-4556-8275-ae00c04985e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:32.404532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-d3c4f34c-9c9f-4556-8275-ae00c04985e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:32.404532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:32.404532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:32.404532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:32.404532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:32.404532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:32.404532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:32.404532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:32.404532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:32.404532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:32.404532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:32.404532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:32.404532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:32.404532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:32.404532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:32.404532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:32.405446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:32.405446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:32.405446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:32.405446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:32.405446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:32.405446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:32.405446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:32.405446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:32.405446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:32.405446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:32.405446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:32.405446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:32.405446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:32.405446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:32.405446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:32.405446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:32.407492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:32.407492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:32.407492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:32.407492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:32.407492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:32.407492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:32.407492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:32.407492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:32.407492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:32.407492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:32.407492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:32.407492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:32.407492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 186/367] 10.222.0.36 () {56 vars in 1102 bytes} [Sun Oct 30 00:35:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:32.416121 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4f34c-9c9f-4556-8275-ae00c04985e4 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:32.434991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:32.466420 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:32.542106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 182/368] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:35:32.554943 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4f34c-9c9f-4556-8275-ae00c04985e4 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:32.584833 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d710e-7eaf-432b-ad08-a94aed3c844f None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:32.594172 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea564e22-76d6-49e8-afbd-c8400b366679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:32.594798 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea564e22-76d6-49e8-afbd-c8400b366679 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:32.595078 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea564e22-76d6-49e8-afbd-c8400b366679 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:32.595351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea564e22-76d6-49e8-afbd-c8400b366679 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:32.596025 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea564e22-76d6-49e8-afbd-c8400b366679 None admin] RBAC: Adding query filter params (name=service) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:32.596360 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea564e22-76d6-49e8-afbd-c8400b366679 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:32.597750 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea564e22-76d6-49e8-afbd-c8400b366679 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:32.607269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 187/369] 10.222.0.36 () {56 vars in 1103 bytes} [Sun Oct 30 00:35:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:32.617351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea564e22-76d6-49e8-afbd-c8400b366679 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:32.700429 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:32.733010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea564e22-76d6-49e8-afbd-c8400b366679 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:32.784517 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d710e-7eaf-432b-ad08-a94aed3c844f None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:32.798792 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0207df-1ae9-4d25-a2c7-68aef0e67daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:32.799589 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0207df-1ae9-4d25-a2c7-68aef0e67daf None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:32.799725 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0207df-1ae9-4d25-a2c7-68aef0e67daf None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:32.800122 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0207df-1ae9-4d25-a2c7-68aef0e67daf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:32.820369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0207df-1ae9-4d25-a2c7-68aef0e67daf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:32.822939 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0207df-1ae9-4d25-a2c7-68aef0e67daf None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:32.853975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-ae0207df-1ae9-4d25-a2c7-68aef0e67daf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:32.853975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:32.853975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:32.853975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:32.853975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:32.853975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:32.853975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:32.853975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:32.853975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:32.853975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:32.853975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:32.853975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:32.853975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:32.853975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:32.853975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:32.853975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:32.854905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:32.854905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:32.854905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:32.854905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:32.854905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:32.854905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:32.854905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:32.854905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:32.854905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:32.854905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:32.854905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:32.854905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:32.854905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:32.854905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:32.854905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:32.854905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:32.854905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:32.856276 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:32.856276 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:32.856276 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:32.856276 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 188/370] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:32.856276 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0207df-1ae9-4d25-a2c7-68aef0e67daf None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:32.857556 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92955ec-e751-4dd6-90d6-52e3868f9fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:32.858275 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92955ec-e751-4dd6-90d6-52e3868f9fa6 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:32.858623 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92955ec-e751-4dd6-90d6-52e3868f9fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:32.858961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92955ec-e751-4dd6-90d6-52e3868f9fa6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:32.871036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92955ec-e751-4dd6-90d6-52e3868f9fa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:32.876497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-e92955ec-e751-4dd6-90d6-52e3868f9fa6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:32.876497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:32.876497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:32.876497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:32.876497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:32.876497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:32.876497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:32.876497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:32.876497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:32.876497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:32.876497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:32.876497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:32.876497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:32.876497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:32.876497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:32.876497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:32.877473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:32.877473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:32.877473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:32.877473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:32.877473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:32.877473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:32.877473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:32.877473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:32.877473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:32.877473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:32.877473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:32.877473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:32.877473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:32.877473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:32.877473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:32.877473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:32.878307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:32.878307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:32.878307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:32.878307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:32.878307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:32.878307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:32.878307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:32.878307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:32.878307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:32.878307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:32.878307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:32.878307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:32.882891 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 183/371] 10.222.0.36 () {56 vars in 1102 bytes} [Sun Oct 30 00:35:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:35:32.884451 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92955ec-e751-4dd6-90d6-52e3868f9fa6 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:32.936036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:33.017932 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92955ec-e751-4dd6-90d6-52e3868f9fa6 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:33.044412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0207df-1ae9-4d25-a2c7-68aef0e67daf None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:33.058254 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf4978-d2c1-423b-a50c-17ecb0872eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:33.059164 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf4978-d2c1-423b-a50c-17ecb0872eb7 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:33.059591 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf4978-d2c1-423b-a50c-17ecb0872eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:33.059997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf4978-d2c1-423b-a50c-17ecb0872eb7 None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:33.060902 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf4978-d2c1-423b-a50c-17ecb0872eb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:33.061340 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf4978-d2c1-423b-a50c-17ecb0872eb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:33.062993 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf4978-d2c1-423b-a50c-17ecb0872eb7 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:33.067242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:33.079270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 184/372] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:33.080184 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba49c2c-47c9-4147-b75f-91d4c912522e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:33.081046 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba49c2c-47c9-4147-b75f-91d4c912522e None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:33.081176 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf4978-d2c1-423b-a50c-17ecb0872eb7 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:33.081627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba49c2c-47c9-4147-b75f-91d4c912522e None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:33.082179 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba49c2c-47c9-4147-b75f-91d4c912522e None admin] PATH_INFO: `/v3/roles/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:33.095000 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba49c2c-47c9-4147-b75f-91d4c912522e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:33.100612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-8ba49c2c-47c9-4147-b75f-91d4c912522e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:33.100612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:33.100612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:33.100612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:33.100612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:33.100612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:33.100612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:33.100612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:33.100612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:33.100612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:33.100612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:33.100612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:33.100612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:33.100612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:33.100612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:33.100612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:33.101527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:33.101527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:33.101527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:33.101527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:33.101527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:33.101527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:33.101527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:33.101527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:33.101527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:33.101527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:33.101527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:33.101527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:33.101527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:33.101527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:33.101527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:33.101527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:33.102356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:33.102356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:33.102356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:33.102356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:33.102356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:33.102356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:33.102356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:33.102356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:33.102356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:33.102356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:33.102356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:33.102356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:33.105131 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 189/373] 10.222.0.36 () {56 vars in 1102 bytes} [Sun Oct 30 00:35:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:33.106544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cd4f3-f770-4b2f-8bd8-6f039947b4f1 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:33.107021 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cd4f3-f770-4b2f-8bd8-6f039947b4f1 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:33.107484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cd4f3-f770-4b2f-8bd8-6f039947b4f1 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:33.108959 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 190/374] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:33.109927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cd4f3-f770-4b2f-8bd8-6f039947b4f1 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:33.192757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:33.200211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf4978-d2c1-423b-a50c-17ecb0872eb7 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:33.237594 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e412b34-fbbc-497c-bb13-cb8b9df6701d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:33.238602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e412b34-fbbc-497c-bb13-cb8b9df6701d None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:33.239088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e412b34-fbbc-497c-bb13-cb8b9df6701d None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:33.239505 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e412b34-fbbc-497c-bb13-cb8b9df6701d None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:33.240344 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e412b34-fbbc-497c-bb13-cb8b9df6701d None admin] RBAC: Adding query filter params (name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:33.240797 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e412b34-fbbc-497c-bb13-cb8b9df6701d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:33.242318 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e412b34-fbbc-497c-bb13-cb8b9df6701d None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:33.242868 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cd4f3-f770-4b2f-8bd8-6f039947b4f1 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:33.253884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 185/375] 10.222.0.36 () {56 vars in 1103 bytes} [Sun Oct 30 00:35:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:33.253884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e412b34-fbbc-497c-bb13-cb8b9df6701d None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:33.288217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83ecd2-711e-4808-b589-923c6f16afc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:33.289142 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83ecd2-711e-4808-b589-923c6f16afc1 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:33.289421 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83ecd2-711e-4808-b589-923c6f16afc1 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:33.289718 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83ecd2-711e-4808-b589-923c6f16afc1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:33.297810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83ecd2-711e-4808-b589-923c6f16afc1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:33.299274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83ecd2-711e-4808-b589-923c6f16afc1 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:33.314528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-df83ecd2-711e-4808-b589-923c6f16afc1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 30 00:35:33.314528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:33.314528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:33.314528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:33.314528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:33.314528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:33.314528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:33.314528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:33.314528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:33.314528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:33.314528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:33.314528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:33.314528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:35:33.314528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:35:33.314528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:35:33.314528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:35:33.315586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:33.315586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:33.315586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:35:33.315586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:33.315586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:35:33.315586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:33.315586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:33.315586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:33.315586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:33.315586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:33.315586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:33.315586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:33.315586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:33.315586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:33.315586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:33.315586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:33.315586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:33.315586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:33.317292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:33.317292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:33.317292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:35:33.317292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:35:33.317292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:35:33.317292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:35:33.317292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:35:33.317292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:35:33.317292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 30 00:35:33.317292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:33.317292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 191/376] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:33.317292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c5307-313a-432d-8eb6-311698ce1627 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:33.317931 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c5307-313a-432d-8eb6-311698ce1627 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:33.317931 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c5307-313a-432d-8eb6-311698ce1627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:33.341504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-509c5307-313a-432d-8eb6-311698ce1627 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:33.342701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-509c5307-313a-432d-8eb6-311698ce1627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:33.378844 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e412b34-fbbc-497c-bb13-cb8b9df6701d None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:33.381345 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:33.415641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd0890-1541-435b-a85c-8861076c5e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:33.416291 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd0890-1541-435b-a85c-8861076c5e75 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:33.416576 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd0890-1541-435b-a85c-8861076c5e75 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:33.416852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd0890-1541-435b-a85c-8861076c5e75 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:33.417523 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd0890-1541-435b-a85c-8861076c5e75 None admin] RBAC: Adding query filter params (name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:33.417854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd0890-1541-435b-a85c-8861076c5e75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:33.419055 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 192/377] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:33.419308 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd0890-1541-435b-a85c-8861076c5e75 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:33.420298 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c5307-313a-432d-8eb6-311698ce1627 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:33.437144 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 186/378] 10.222.0.36 () {56 vars in 1103 bytes} [Sun Oct 30 00:35:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:33.438081 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd0890-1541-435b-a85c-8861076c5e75 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:33.463753 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:33.549576 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c5307-313a-432d-8eb6-311698ce1627 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:33.587817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd0890-1541-435b-a85c-8861076c5e75 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:33.589826 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126179a3-4d9f-4a86-bc0e-0ed0142edaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:33.590827 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126179a3-4d9f-4a86-bc0e-0ed0142edaae None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:33.591186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126179a3-4d9f-4a86-bc0e-0ed0142edaae None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:33.591594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126179a3-4d9f-4a86-bc0e-0ed0142edaae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:33.609346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126179a3-4d9f-4a86-bc0e-0ed0142edaae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:33.611720 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126179a3-4d9f-4a86-bc0e-0ed0142edaae None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:33.634623 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee7bdb-f57e-40db-befc-949a00eb062d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:33.635514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee7bdb-f57e-40db-befc-949a00eb062d None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:33.635976 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee7bdb-f57e-40db-befc-949a00eb062d None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:33.636430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee7bdb-f57e-40db-befc-949a00eb062d None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:33.637347 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee7bdb-f57e-40db-befc-949a00eb062d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:33.637828 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee7bdb-f57e-40db-befc-949a00eb062d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:33.638246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-126179a3-4d9f-4a86-bc0e-0ed0142edaae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:33.638246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:33.638246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:33.638246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:33.638246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:33.638246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:33.638246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:33.638246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:33.638246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:33.638246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:33.638246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:33.638246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:33.638246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:33.638246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:33.638246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:33.638246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:33.639141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:33.639141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:33.639141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:33.639141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:33.639141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:33.639141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:33.639141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:33.639141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:33.639141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:33.639141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:33.639141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:33.639141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:33.639141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:33.639141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:33.639141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:33.639141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:33.639141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:33.644748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:33.644748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:33.644748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:33.644748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 193/379] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:33.644748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126179a3-4d9f-4a86-bc0e-0ed0142edaae None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:33.647108 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee7bdb-f57e-40db-befc-949a00eb062d None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:33.676914 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:33.703088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 187/380] 10.222.0.36 () {56 vars in 1137 bytes} [Sun Oct 30 00:35:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:33.704547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee7bdb-f57e-40db-befc-949a00eb062d None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:33.761289 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126179a3-4d9f-4a86-bc0e-0ed0142edaae None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:33.800282 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885dc3f-561a-47a1-a41c-2da9028be415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:33.800282 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885dc3f-561a-47a1-a41c-2da9028be415 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:33.801094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885dc3f-561a-47a1-a41c-2da9028be415 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:33.801094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885dc3f-561a-47a1-a41c-2da9028be415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:33.803128 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:33.813891 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4885dc3f-561a-47a1-a41c-2da9028be415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:33.815452 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4885dc3f-561a-47a1-a41c-2da9028be415 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:33.826257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee7bdb-f57e-40db-befc-949a00eb062d None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:33.830160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-4885dc3f-561a-47a1-a41c-2da9028be415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:33.830160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:33.830160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:33.830160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:33.830160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:33.830160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:33.830160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:33.830160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:33.830160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:33.830160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:33.830160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:33.830160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:33.830160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:33.830160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:33.830160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:33.830160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:33.831353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:33.831353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:33.831353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:33.831353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:33.831353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:33.831353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:33.831353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:33.831353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:33.831353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:33.831353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:33.831353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:33.831353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:33.831353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:33.831353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:33.831353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:33.831353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:33.831353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:33.832651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:33.832651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:33.832651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:33.832651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 194/381] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:33.832946 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9da23-a03e-4d15-b546-4cf5c2969aa9 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:33.833090 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9da23-a03e-4d15-b546-4cf5c2969aa9 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:33.833442 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9da23-a03e-4d15-b546-4cf5c2969aa9 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:33.834939 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 195/382] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:33.835429 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9da23-a03e-4d15-b546-4cf5c2969aa9 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:33.864329 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9e427-1d5f-4594-ad2d-ed7455bfa4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:33.864939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9e427-1d5f-4594-ad2d-ed7455bfa4d6 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:33.865213 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9e427-1d5f-4594-ad2d-ed7455bfa4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:33.865490 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9e427-1d5f-4594-ad2d-ed7455bfa4d6 None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:33.866128 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9e427-1d5f-4594-ad2d-ed7455bfa4d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:33.866426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9e427-1d5f-4594-ad2d-ed7455bfa4d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:33.867817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9e427-1d5f-4594-ad2d-ed7455bfa4d6 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:33.880618 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 188/383] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:33.881472 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9e427-1d5f-4594-ad2d-ed7455bfa4d6 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:33.932396 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:33.960100 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9da23-a03e-4d15-b546-4cf5c2969aa9 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:33.994336 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9e427-1d5f-4594-ad2d-ed7455bfa4d6 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:34.005081 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc941f7-6f24-4851-90bc-c137734f9144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:34.005730 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc941f7-6f24-4851-90bc-c137734f9144 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:34.006007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc941f7-6f24-4851-90bc-c137734f9144 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:34.006279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc941f7-6f24-4851-90bc-c137734f9144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:34.022503 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc941f7-6f24-4851-90bc-c137734f9144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:34.023983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc941f7-6f24-4851-90bc-c137734f9144 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:34.029158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f15f1-461a-4ee7-b4de-1c8e31348860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:34.029794 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f15f1-461a-4ee7-b4de-1c8e31348860 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:34.030071 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f15f1-461a-4ee7-b4de-1c8e31348860 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:34.030342 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f15f1-461a-4ee7-b4de-1c8e31348860 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:34.038126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:34.044534 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54f15f1-461a-4ee7-b4de-1c8e31348860 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:34.044780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-6cc941f7-6f24-4851-90bc-c137734f9144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:34.044780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:34.044780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:34.044780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:34.044780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:34.044780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:34.044780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:34.044780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:34.044780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:34.044780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:34.044780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:34.044780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:34.044780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:34.044780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:34.044780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:34.044780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:34.045810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:34.045810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:34.045810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:34.045810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:34.045810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:34.045810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:34.045810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:34.045810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:34.045810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:34.045810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:34.045810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:34.045810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:34.045810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:34.045810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:34.045810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:34.045810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:34.045810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:34.052481 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:34.052481 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:34.052481 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:34.052481 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 196/384] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:34.052481 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc941f7-6f24-4851-90bc-c137734f9144 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:34.052775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-a54f15f1-461a-4ee7-b4de-1c8e31348860 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:34.052775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:34.052775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:34.052775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:34.052775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:34.052775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:34.052775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:34.052775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:34.052775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:34.052775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:34.052775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:34.052775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:34.052775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:34.052775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:34.052775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:34.052775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:34.053676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:34.053676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:34.053676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:34.053676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:34.053676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:34.053676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:34.053676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:34.053676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:34.053676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:34.053676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:34.053676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:34.053676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:34.053676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:34.053676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:34.053676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:34.053676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:34.054482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:34.054482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:34.054482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:34.054482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:34.054482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:34.054482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:34.054482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:34.054482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:34.054482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:34.054482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:34.054482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:34.054482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:34.054482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 189/385] 10.222.0.36 () {56 vars in 1102 bytes} [Sun Oct 30 00:35:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:34.054482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2b1a4-1c38-4bb7-9e2b-a436bfce965c None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:34.055518 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2b1a4-1c38-4bb7-9e2b-a436bfce965c None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:34.055518 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2b1a4-1c38-4bb7-9e2b-a436bfce965c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:34.071137 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-6cb2b1a4-1c38-4bb7-9e2b-a436bfce965c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:34.072383 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-6cb2b1a4-1c38-4bb7-9e2b-a436bfce965c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:34.142941 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 190/386] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:34.143986 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2b1a4-1c38-4bb7-9e2b-a436bfce965c None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:34.168169 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc941f7-6f24-4851-90bc-c137734f9144 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:34.198917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:34.210456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50abd80-840a-49bc-a877-e71c05dfd827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:34.211377 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50abd80-840a-49bc-a877-e71c05dfd827 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:34.211663 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50abd80-840a-49bc-a877-e71c05dfd827 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:34.212539 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50abd80-840a-49bc-a877-e71c05dfd827 None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:34.212539 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50abd80-840a-49bc-a877-e71c05dfd827 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:34.212539 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50abd80-840a-49bc-a877-e71c05dfd827 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:34.212539 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50abd80-840a-49bc-a877-e71c05dfd827 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:34.229664 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 197/387] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:34.236404 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50abd80-840a-49bc-a877-e71c05dfd827 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:34.263221 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2b1a4-1c38-4bb7-9e2b-a436bfce965c None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:34.298863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:34.309609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55073c4d-1458-4b2d-997d-ec6d36e4b856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:34.310411 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55073c4d-1458-4b2d-997d-ec6d36e4b856 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:34.310692 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55073c4d-1458-4b2d-997d-ec6d36e4b856 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:34.310962 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55073c4d-1458-4b2d-997d-ec6d36e4b856 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:34.319334 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55073c4d-1458-4b2d-997d-ec6d36e4b856 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:34.320836 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55073c4d-1458-4b2d-997d-ec6d36e4b856 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:34.334662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-55073c4d-1458-4b2d-997d-ec6d36e4b856 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 30 00:35:34.334662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:34.334662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:34.334662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:34.334662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:34.334662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:34.334662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:34.334662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:34.334662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:34.334662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:34.334662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:34.334662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:34.334662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:35:34.334662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:35:34.334662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:35:34.334662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:35:34.335801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:34.335801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:34.335801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:35:34.335801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:34.335801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:35:34.335801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:34.335801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:34.335801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:34.335801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:34.335801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:34.335801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:34.335801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:34.335801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:34.335801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:34.335801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:34.335801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:34.335801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:34.335801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:34.336840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:34.336840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:34.336840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:35:34.336840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:35:34.336840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:35:34.336840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:35:34.336840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:35:34.336840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:35:34.336840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 30 00:35:34.336840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:34.336840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 191/388] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:34.338055 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55073c4d-1458-4b2d-997d-ec6d36e4b856 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:34.369406 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50abd80-840a-49bc-a877-e71c05dfd827 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:34.411920 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a623712-8ddb-410c-bada-738850900c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:34.412533 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a623712-8ddb-410c-bada-738850900c1f None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:34.412943 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a623712-8ddb-410c-bada-738850900c1f None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:34.413082 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a623712-8ddb-410c-bada-738850900c1f None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:34.413981 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a623712-8ddb-410c-bada-738850900c1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:34.414280 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a623712-8ddb-410c-bada-738850900c1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:34.415755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a623712-8ddb-410c-bada-738850900c1f None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:34.429161 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 198/389] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:34.430018 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a623712-8ddb-410c-bada-738850900c1f None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:34.469810 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55073c4d-1458-4b2d-997d-ec6d36e4b856 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:34.490158 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:34.506042 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfe8ca-7133-4858-b696-ceffcfb11868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:34.506663 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfe8ca-7133-4858-b696-ceffcfb11868 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:34.506941 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfe8ca-7133-4858-b696-ceffcfb11868 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:34.507216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfe8ca-7133-4858-b696-ceffcfb11868 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:34.507881 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfe8ca-7133-4858-b696-ceffcfb11868 None admin] RBAC: Adding query filter params (name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:34.508212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfe8ca-7133-4858-b696-ceffcfb11868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:34.510377 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfe8ca-7133-4858-b696-ceffcfb11868 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:34.518751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 192/390] 10.222.0.36 () {56 vars in 1103 bytes} [Sun Oct 30 00:35:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:34.523391 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfe8ca-7133-4858-b696-ceffcfb11868 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:34.545142 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a623712-8ddb-410c-bada-738850900c1f None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:34.579069 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:34.581249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c272751-0d1c-46bd-a45f-4b3db1dff836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:34.581915 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c272751-0d1c-46bd-a45f-4b3db1dff836 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:34.582446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c272751-0d1c-46bd-a45f-4b3db1dff836 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:34.582446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c272751-0d1c-46bd-a45f-4b3db1dff836 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:34.591137 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c272751-0d1c-46bd-a45f-4b3db1dff836 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:34.592642 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c272751-0d1c-46bd-a45f-4b3db1dff836 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:34.604594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-8c272751-0d1c-46bd-a45f-4b3db1dff836 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:35:34.604594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:34.604594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:34.604594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:34.604594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:34.604594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:34.604594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:34.604594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:34.604594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:34.604594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:34.604594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:34.604594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:34.604594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:35:34.604594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:35:34.604594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:35:34.604594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:35:34.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:34.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:34.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:35:34.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:34.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:35:34.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:34.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:34.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:34.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:34.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:34.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:34.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:34.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:34.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:34.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:34.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:34.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:34.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:34.606598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:34.606598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:34.606598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:35:34.606598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:35:34.606598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:35:34.606598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:35:34.606598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:35:34.606598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:35:34.606598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:35:34.606598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:34.606598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 199/391] 10.222.0.36 () {56 vars in 1093 bytes} [Sun Oct 30 00:35:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:34.606598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c272751-0d1c-46bd-a45f-4b3db1dff836 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:34.638538 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfe8ca-7133-4858-b696-ceffcfb11868 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:34.675867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d58f6-9ce4-4cb1-9ab0-a9f7c8993f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:34.676506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d58f6-9ce4-4cb1-9ab0-a9f7c8993f82 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:34.676786 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d58f6-9ce4-4cb1-9ab0-a9f7c8993f82 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:34.677056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d58f6-9ce4-4cb1-9ab0-a9f7c8993f82 None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:34.677792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d58f6-9ce4-4cb1-9ab0-a9f7c8993f82 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:34.678093 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d58f6-9ce4-4cb1-9ab0-a9f7c8993f82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:34.679545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d58f6-9ce4-4cb1-9ab0-a9f7c8993f82 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:34.699840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 193/392] 10.222.0.36 () {56 vars in 1137 bytes} [Sun Oct 30 00:35:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:34.700729 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d58f6-9ce4-4cb1-9ab0-a9f7c8993f82 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:34.729281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c272751-0d1c-46bd-a45f-4b3db1dff836 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:34.738670 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:34.767817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0868666-cf48-43ed-9eff-808c1ee3df2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:34.768594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0868666-cf48-43ed-9eff-808c1ee3df2e None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:34.768882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0868666-cf48-43ed-9eff-808c1ee3df2e None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:34.769159 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0868666-cf48-43ed-9eff-808c1ee3df2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:34.780424 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0868666-cf48-43ed-9eff-808c1ee3df2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:34.781620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0868666-cf48-43ed-9eff-808c1ee3df2e None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:34.796812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-e0868666-cf48-43ed-9eff-808c1ee3df2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:34.796812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:34.796812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:34.796812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:34.796812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:34.796812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:34.796812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:34.796812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:34.796812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:34.796812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:34.796812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:34.796812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:34.796812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:34.796812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:34.796812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:34.796812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:34.797978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:34.797978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:34.797978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:34.797978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:34.797978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:34.797978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:34.797978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:34.797978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:34.797978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:34.797978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:34.797978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:34.797978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:34.797978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:34.797978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:34.797978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:34.797978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:34.797978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:34.799008 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:34.799008 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:34.799008 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:34.799008 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 200/393] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:34.803911 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0868666-cf48-43ed-9eff-808c1ee3df2e None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:34.819824 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d58f6-9ce4-4cb1-9ab0-a9f7c8993f82 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:34.831013 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:34.856697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd118359-b439-4e12-94ec-1591050df3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:34.861105 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd118359-b439-4e12-94ec-1591050df3e5 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:34.861105 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd118359-b439-4e12-94ec-1591050df3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:34.861105 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd118359-b439-4e12-94ec-1591050df3e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:34.865063 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd118359-b439-4e12-94ec-1591050df3e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:34.866633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd118359-b439-4e12-94ec-1591050df3e5 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:34.879964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-bd118359-b439-4e12-94ec-1591050df3e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:34.879964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:34.879964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:34.879964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:34.879964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:34.879964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:34.879964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:34.879964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:34.879964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:34.879964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:34.879964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:34.879964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:34.879964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:35:34.879964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:35:34.879964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:35:34.880864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:35:34.880864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:34.880864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:34.880864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:34.880864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:34.880864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:34.880864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:34.880864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:34.880864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:34.880864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:34.880864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:34.880864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:34.880864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:34.880864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:34.880864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:34.880864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:34.880864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:34.881731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:35:34.881731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:35:34.881731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:35:34.881731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:35:34.881731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:35:34.881731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:35:34.881731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:34.881731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:34.881731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 194/394] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:34.881731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd118359-b439-4e12-94ec-1591050df3e5 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:34.918532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0868666-cf48-43ed-9eff-808c1ee3df2e None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:34.954137 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9825e8-4fe5-4f16-b324-3faf8fe05bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:34.954801 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9825e8-4fe5-4f16-b324-3faf8fe05bda None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:34.955095 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9825e8-4fe5-4f16-b324-3faf8fe05bda None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:34.955384 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9825e8-4fe5-4f16-b324-3faf8fe05bda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:34.966660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9825e8-4fe5-4f16-b324-3faf8fe05bda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:34.968249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9825e8-4fe5-4f16-b324-3faf8fe05bda None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:34.983213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-8e9825e8-4fe5-4f16-b324-3faf8fe05bda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:34.983213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:34.983213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:34.983213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:34.983213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:34.983213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:34.983213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:34.983213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:34.983213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:34.983213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:34.983213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:34.983213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:34.983213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:34.983213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:34.983213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:34.983213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:34.984259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:34.984259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:34.984259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:34.984259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:34.984259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:34.984259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:34.984259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:34.984259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:34.984259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:34.984259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:34.984259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:34.984259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:34.984259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:34.984259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:34.984259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:34.984259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:34.984259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:34.990064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:34.990064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:34.990064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:34.990064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 201/395] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:34.990064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9825e8-4fe5-4f16-b324-3faf8fe05bda None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:34.990419 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd118359-b439-4e12-94ec-1591050df3e5 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:35.002773 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:35.026756 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b84398-d13b-4f3d-b545-8125984f2dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:35.027364 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b84398-d13b-4f3d-b545-8125984f2dee None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:35.027652 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b84398-d13b-4f3d-b545-8125984f2dee None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:35.027923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b84398-d13b-4f3d-b545-8125984f2dee None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:35.028693 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b84398-d13b-4f3d-b545-8125984f2dee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:35.028999 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b84398-d13b-4f3d-b545-8125984f2dee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:35.030438 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b84398-d13b-4f3d-b545-8125984f2dee None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:35.048164 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 195/396] 10.222.0.36 () {56 vars in 1133 bytes} [Sun Oct 30 00:35:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:35.048991 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b84398-d13b-4f3d-b545-8125984f2dee None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:35.077340 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:35.094052 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9825e8-4fe5-4f16-b324-3faf8fe05bda None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:35.129829 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6f79f-37ff-4434-abe1-d822565fbefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:35.130465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6f79f-37ff-4434-abe1-d822565fbefe None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:35.130748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6f79f-37ff-4434-abe1-d822565fbefe None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:35.131021 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6f79f-37ff-4434-abe1-d822565fbefe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:35.143211 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6f79f-37ff-4434-abe1-d822565fbefe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:35.144790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6f79f-37ff-4434-abe1-d822565fbefe None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:35.159936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-07a6f79f-37ff-4434-abe1-d822565fbefe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:35.159936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:35.159936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:35.159936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:35.159936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:35.159936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:35.159936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:35.159936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:35.159936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:35.159936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:35.159936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:35.159936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:35.159936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:35.159936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:35.159936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:35.159936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:35.161249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:35.161249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:35.161249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:35.161249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:35.161249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:35.161249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:35.161249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:35.161249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:35.161249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:35.161249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:35.161249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:35.161249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:35.161249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:35.161249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:35.161249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:35.161249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:35.161249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:35.162345 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:35.162345 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:35.162345 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:35.162345 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 202/397] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:35.162345 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6f79f-37ff-4434-abe1-d822565fbefe None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:35.170639 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b84398-d13b-4f3d-b545-8125984f2dee None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:35.206412 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064a4c3-6ffb-49bb-92e7-34982a5e814b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:35.207022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064a4c3-6ffb-49bb-92e7-34982a5e814b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:35.207297 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064a4c3-6ffb-49bb-92e7-34982a5e814b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:35.207565 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064a4c3-6ffb-49bb-92e7-34982a5e814b None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:35.208231 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5064a4c3-6ffb-49bb-92e7-34982a5e814b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:35.208533 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5064a4c3-6ffb-49bb-92e7-34982a5e814b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:35.209897 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5064a4c3-6ffb-49bb-92e7-34982a5e814b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:35.221294 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 196/398] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:35.222201 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064a4c3-6ffb-49bb-92e7-34982a5e814b None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:35.230688 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:35.269252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6f79f-37ff-4434-abe1-d822565fbefe None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:35.306556 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec8568-548f-4f2e-9b92-7a7a1fa198b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:35.307188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec8568-548f-4f2e-9b92-7a7a1fa198b2 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:35.307468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec8568-548f-4f2e-9b92-7a7a1fa198b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:35.307746 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec8568-548f-4f2e-9b92-7a7a1fa198b2 None admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:35.308466 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec8568-548f-4f2e-9b92-7a7a1fa198b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:35.308770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec8568-548f-4f2e-9b92-7a7a1fa198b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:35.310201 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec8568-548f-4f2e-9b92-7a7a1fa198b2 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:35.312294 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:35.327953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 203/399] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:35:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:35.328261 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064a4c3-6ffb-49bb-92e7-34982a5e814b None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:35.328820 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec8568-548f-4f2e-9b92-7a7a1fa198b2 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:35.373513 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a4fdb-f32b-4ffb-ba77-d4641ffc522a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:35.374285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a4fdb-f32b-4ffb-ba77-d4641ffc522a None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:35.374661 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a4fdb-f32b-4ffb-ba77-d4641ffc522a None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:35.375033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a4fdb-f32b-4ffb-ba77-d4641ffc522a None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:35.375780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a4fdb-f32b-4ffb-ba77-d4641ffc522a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:35.376232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a4fdb-f32b-4ffb-ba77-d4641ffc522a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:35.377709 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a4fdb-f32b-4ffb-ba77-d4641ffc522a None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:35.391314 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 197/400] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:35.392165 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a4fdb-f32b-4ffb-ba77-d4641ffc522a None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:35.451951 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec8568-548f-4f2e-9b92-7a7a1fa198b2 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:35.476667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:35.490212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108aa94-3d88-4262-848a-4ed7f2803ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:35.490848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108aa94-3d88-4262-848a-4ed7f2803ed5 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:35.491133 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108aa94-3d88-4262-848a-4ed7f2803ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:35.491403 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108aa94-3d88-4262-848a-4ed7f2803ed5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:35.494701 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a4fdb-f32b-4ffb-ba77-d4641ffc522a None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:35.503058 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7108aa94-3d88-4262-848a-4ed7f2803ed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:35.504591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7108aa94-3d88-4262-848a-4ed7f2803ed5 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:35.520353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-7108aa94-3d88-4262-848a-4ed7f2803ed5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:35.520353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:35.520353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:35.520353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:35.520353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:35.520353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:35.520353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:35.520353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:35.520353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:35.520353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:35.520353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:35.520353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:35.520353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:35.520353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:35.520353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:35.520353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:35.521469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:35.521469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:35.521469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:35.521469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:35.521469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:35.521469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:35.521469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:35.521469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:35.521469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:35.521469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:35.521469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:35.521469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:35.521469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:35.521469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:35.521469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:35.521469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:35.521469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:35.522507 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:35.522507 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:35.522507 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:35.526755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 204/401] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:35.527419 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108aa94-3d88-4262-848a-4ed7f2803ed5 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:35.532596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67059bc7-ea06-402a-a07b-334d1d9b9b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:35.533200 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67059bc7-ea06-402a-a07b-334d1d9b9b97 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:35.533473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67059bc7-ea06-402a-a07b-334d1d9b9b97 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:35.533740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67059bc7-ea06-402a-a07b-334d1d9b9b97 None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:35.534374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67059bc7-ea06-402a-a07b-334d1d9b9b97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:35.534667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67059bc7-ea06-402a-a07b-334d1d9b9b97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:35.536036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67059bc7-ea06-402a-a07b-334d1d9b9b97 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:35.554059 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 198/402] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:35.554059 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67059bc7-ea06-402a-a07b-334d1d9b9b97 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:35.561245 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:35.639650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108aa94-3d88-4262-848a-4ed7f2803ed5 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:35.662955 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67059bc7-ea06-402a-a07b-334d1d9b9b97 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:35.675857 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22904d9-d8f7-402d-853b-a7d6cc35ef80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:35.676498 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22904d9-d8f7-402d-853b-a7d6cc35ef80 None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:35.676782 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22904d9-d8f7-402d-853b-a7d6cc35ef80 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:35.677057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22904d9-d8f7-402d-853b-a7d6cc35ef80 None admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:35.677735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22904d9-d8f7-402d-853b-a7d6cc35ef80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:35.679230 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22904d9-d8f7-402d-853b-a7d6cc35ef80 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:35.697792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed787e3-2bbf-48e4-99a9-f9d89ea8bd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:35.698463 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed787e3-2bbf-48e4-99a9-f9d89ea8bd5f None admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:35.698744 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed787e3-2bbf-48e4-99a9-f9d89ea8bd5f None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:35.699018 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed787e3-2bbf-48e4-99a9-f9d89ea8bd5f None admin] PATH_INFO: `/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/7cf9451f2ef7478589b4a4278cb03252/roles/9a8de43206fa4a8fa4901488bf86f9f5` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:35.700279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 205/403] 10.222.0.36 () {60 vars in 1132 bytes} [Sun Oct 30 00:35:35 2022] POST /identity/v3/users => generated 247 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:35:35.701111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22904d9-d8f7-402d-853b-a7d6cc35ef80 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:35.724989 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed787e3-2bbf-48e4-99a9-f9d89ea8bd5f None admin] RBAC: Authorizing `identity:create_grant(project_id=ab3a0e5fc5a344739b58ec8313639496, user_id=7cf9451f2ef7478589b4a4278cb03252, role_id=9a8de43206fa4a8fa4901488bf86f9f5)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:35.726630 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed787e3-2bbf-48e4-99a9-f9d89ea8bd5f None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:35.743054 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 199/404] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:35:35 2022] PUT /identity/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/7cf9451f2ef7478589b4a4278cb03252/roles/9a8de43206fa4a8fa4901488bf86f9f5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:35:35.752404 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed787e3-2bbf-48e4-99a9-f9d89ea8bd5f None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:35.776575 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:35.829637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:35.891416 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22904d9-d8f7-402d-853b-a7d6cc35ef80 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:35.905386 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed787e3-2bbf-48e4-99a9-f9d89ea8bd5f None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:35.940179 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c83e8-f73d-4331-a441-b87a3f8e8951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:35.940842 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c83e8-f73d-4331-a441-b87a3f8e8951 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:35.941126 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c83e8-f73d-4331-a441-b87a3f8e8951 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:35.941402 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c83e8-f73d-4331-a441-b87a3f8e8951 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:35.952518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c83e8-f73d-4331-a441-b87a3f8e8951 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:35.953722 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243dbb22-f153-4836-be6c-034a09f48eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:35.954264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c83e8-f73d-4331-a441-b87a3f8e8951 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:35.954560 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243dbb22-f153-4836-be6c-034a09f48eee None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:35.954961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243dbb22-f153-4836-be6c-034a09f48eee None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:35.955239 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243dbb22-f153-4836-be6c-034a09f48eee None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:35.955904 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243dbb22-f153-4836-be6c-034a09f48eee None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:35.956238 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243dbb22-f153-4836-be6c-034a09f48eee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:35.957640 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243dbb22-f153-4836-be6c-034a09f48eee None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:35.968500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-440c83e8-f73d-4331-a441-b87a3f8e8951 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 30 00:35:35.968500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:35.968500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:35.968500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:35.968500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:35.968500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:35.968500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:35.968500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:35.968500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:35.968500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:35.968500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:35.968500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:35.968500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:35:35.968500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:35:35.968500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:35:35.968500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:35:35.969465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:35.969465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:35.969465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:35:35.969465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:35.969465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:35:35.969465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:35.969465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:35.969465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:35.969465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:35.969465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:35.969465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:35.969465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:35.969465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:35.969465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:35.969465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:35.969465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:35.969465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:35.969465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:35.972615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:35.972615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:35.972615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:35:35.972615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:35:35.972615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:35:35.972615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:35:35.972615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:35:35.972615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:35:35.972615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 30 00:35:35.972615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:35.972615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 206/405] 10.222.0.36 () {56 vars in 1102 bytes} [Sun Oct 30 00:35:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:35.972615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c83e8-f73d-4331-a441-b87a3f8e8951 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:35.974129 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 200/406] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:35.976398 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243dbb22-f153-4836-be6c-034a09f48eee None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:36.099650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:36.101331 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243dbb22-f153-4836-be6c-034a09f48eee None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:36.112603 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c83e8-f73d-4331-a441-b87a3f8e8951 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:36.119557 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:36.139432 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19819fa5-d63a-4352-b3ad-62fcd8790b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:36.140058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19819fa5-d63a-4352-b3ad-62fcd8790b3e None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:36.140407 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19819fa5-d63a-4352-b3ad-62fcd8790b3e None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:36.140685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19819fa5-d63a-4352-b3ad-62fcd8790b3e None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:36.141420 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19819fa5-d63a-4352-b3ad-62fcd8790b3e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:36.141728 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19819fa5-d63a-4352-b3ad-62fcd8790b3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:36.143187 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19819fa5-d63a-4352-b3ad-62fcd8790b3e None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:36.154159 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956db753-ecd1-47a7-875e-d6edfa82cd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:36.155084 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956db753-ecd1-47a7-875e-d6edfa82cd50 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:36.155547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956db753-ecd1-47a7-875e-d6edfa82cd50 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:36.155995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956db753-ecd1-47a7-875e-d6edfa82cd50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:36.163052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 201/407] 10.222.0.36 () {56 vars in 1139 bytes} [Sun Oct 30 00:35:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:36.164237 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19819fa5-d63a-4352-b3ad-62fcd8790b3e None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:36.164445 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956db753-ecd1-47a7-875e-d6edfa82cd50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:36.166143 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956db753-ecd1-47a7-875e-d6edfa82cd50 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:36.179571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-956db753-ecd1-47a7-875e-d6edfa82cd50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:36.179571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:36.179571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:36.179571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:36.179571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:36.179571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:36.179571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:36.179571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:36.179571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:36.179571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:36.179571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:36.179571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:36.179571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:35:36.179571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:35:36.179571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:35:36.180689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:35:36.180689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:36.180689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:36.180689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:36.180689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:36.180689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:36.180689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:36.180689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:36.180689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:36.180689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:36.180689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:36.180689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:36.180689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:36.180689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:36.180689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:36.180689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:36.180689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:36.181652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:35:36.181652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:35:36.181652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:35:36.181652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:35:36.181652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:35:36.181652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:35:36.181652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:36.181652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:36.183924 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 207/408] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:36.185185 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956db753-ecd1-47a7-875e-d6edfa82cd50 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:36.283535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19819fa5-d63a-4352-b3ad-62fcd8790b3e None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:36.306114 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956db753-ecd1-47a7-875e-d6edfa82cd50 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:36.320382 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a3f8d-6bbb-41bf-b650-cec32c8af23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:36.321028 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a3f8d-6bbb-41bf-b650-cec32c8af23b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:36.321308 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a3f8d-6bbb-41bf-b650-cec32c8af23b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:36.321808 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a3f8d-6bbb-41bf-b650-cec32c8af23b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:36.329104 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a3f8d-6bbb-41bf-b650-cec32c8af23b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:36.330708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a3f8d-6bbb-41bf-b650-cec32c8af23b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:36.342886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-257a3f8d-6bbb-41bf-b650-cec32c8af23b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:36.342886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:36.342886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:36.342886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:36.342886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:36.342886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:36.342886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:36.342886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:36.342886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:36.342886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:36.342886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:36.342886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:36.342886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:35:36.342886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:35:36.342886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:35:36.343768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:35:36.343768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:36.343768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:36.343768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:36.343768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:36.343768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:36.343768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:36.343768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:36.343768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:36.343768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:36.343768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:36.343768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:36.343768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:36.343768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:36.343768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:36.343768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:36.343768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:36.344664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:35:36.344664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:35:36.344664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:35:36.344664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:35:36.344664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:35:36.344664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:35:36.344664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:36.344664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:36.348434 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 202/409] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:36.349337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a3f8d-6bbb-41bf-b650-cec32c8af23b None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:36.349540 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93289676-4efa-4b50-800a-190ee0218865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:36.350576 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93289676-4efa-4b50-800a-190ee0218865 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:36.351038 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93289676-4efa-4b50-800a-190ee0218865 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:36.351501 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93289676-4efa-4b50-800a-190ee0218865 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:36.365688 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93289676-4efa-4b50-800a-190ee0218865 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:36.367662 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93289676-4efa-4b50-800a-190ee0218865 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:36.384721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-93289676-4efa-4b50-800a-190ee0218865 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:36.384721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:36.384721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:36.384721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:36.384721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:36.384721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:36.384721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:36.384721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:36.384721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:36.384721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:36.384721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:36.384721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:36.384721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:36.384721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:36.384721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:36.384721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:36.385701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:36.385701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:36.385701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:36.385701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:36.385701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:36.385701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:36.385701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:36.385701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:36.385701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:36.385701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:36.385701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:36.385701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:36.385701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:36.385701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:36.385701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:36.385701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:36.385701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:36.386579 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:36.386579 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:36.386579 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:36.387439 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:36.389827 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 208/410] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:36.391019 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93289676-4efa-4b50-800a-190ee0218865 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:36.395995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:36.460262 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a3f8d-6bbb-41bf-b650-cec32c8af23b None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:36.497643 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f884407-0f56-4cf4-8935-50c2d00db464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:36.498257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f884407-0f56-4cf4-8935-50c2d00db464 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:36.498534 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f884407-0f56-4cf4-8935-50c2d00db464 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:36.498802 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f884407-0f56-4cf4-8935-50c2d00db464 None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:36.499556 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f884407-0f56-4cf4-8935-50c2d00db464 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:36.499801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f884407-0f56-4cf4-8935-50c2d00db464 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:36.502070 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f884407-0f56-4cf4-8935-50c2d00db464 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:36.528489 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 203/411] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:35:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:36.529488 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f884407-0f56-4cf4-8935-50c2d00db464 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:36.531085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93289676-4efa-4b50-800a-190ee0218865 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:36.582601 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4fba2-1064-442d-a72c-fc50eaf1343d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:36.583754 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4fba2-1064-442d-a72c-fc50eaf1343d None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:36.584209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4fba2-1064-442d-a72c-fc50eaf1343d None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:36.584866 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4fba2-1064-442d-a72c-fc50eaf1343d None admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:36.586136 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4fba2-1064-442d-a72c-fc50eaf1343d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:36.586619 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4fba2-1064-442d-a72c-fc50eaf1343d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:36.588890 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4fba2-1064-442d-a72c-fc50eaf1343d None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:36.603509 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 209/412] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:35:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:36.609301 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4fba2-1064-442d-a72c-fc50eaf1343d None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:36.647009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:36.658071 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f884407-0f56-4cf4-8935-50c2d00db464 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:36.679167 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:36.706927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88cd5b3-5bce-4c49-8abe-33b7ce6b4621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:36.707780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cd5b3-5bce-4c49-8abe-33b7ce6b4621 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:36.708236 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cd5b3-5bce-4c49-8abe-33b7ce6b4621 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:36.709108 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cd5b3-5bce-4c49-8abe-33b7ce6b4621 None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:36.709926 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88cd5b3-5bce-4c49-8abe-33b7ce6b4621 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:36.710664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88cd5b3-5bce-4c49-8abe-33b7ce6b4621 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:36.712743 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88cd5b3-5bce-4c49-8abe-33b7ce6b4621 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:36.721835 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4fba2-1064-442d-a72c-fc50eaf1343d None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:36.729217 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 204/413] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:36.730339 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88cd5b3-5bce-4c49-8abe-33b7ce6b4621 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:36.763651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc92a32-082f-42ae-b682-d0775449d5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:36.764424 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc92a32-082f-42ae-b682-d0775449d5c2 None admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:36.764784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc92a32-082f-42ae-b682-d0775449d5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:36.765177 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc92a32-082f-42ae-b682-d0775449d5c2 None admin] PATH_INFO: `/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/2dbe6b3bdfb3405589b8ce5aebebb93e/roles/9a8de43206fa4a8fa4901488bf86f9f5` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:36.782783 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc92a32-082f-42ae-b682-d0775449d5c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab3a0e5fc5a344739b58ec8313639496, user_id=2dbe6b3bdfb3405589b8ce5aebebb93e, role_id=9a8de43206fa4a8fa4901488bf86f9f5)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:36.784542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc92a32-082f-42ae-b682-d0775449d5c2 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:36.808295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 210/414] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:35:36 2022] PUT /identity/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/2dbe6b3bdfb3405589b8ce5aebebb93e/roles/9a8de43206fa4a8fa4901488bf86f9f5 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:35:36.809411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc92a32-082f-42ae-b682-d0775449d5c2 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:36.867103 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88cd5b3-5bce-4c49-8abe-33b7ce6b4621 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:36.914359 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b5bce-fdf4-4769-a51e-259a38c8f95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:36.915938 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b5bce-fdf4-4769-a51e-259a38c8f95a None admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:36.916470 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b5bce-fdf4-4769-a51e-259a38c8f95a None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:36.916954 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b5bce-fdf4-4769-a51e-259a38c8f95a None admin] PATH_INFO: `/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/adbba7be39a64230bab160ce77ce7586/roles/9a8de43206fa4a8fa4901488bf86f9f5` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:36.935835 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b5bce-fdf4-4769-a51e-259a38c8f95a None admin] RBAC: Authorizing `identity:create_grant(project_id=ab3a0e5fc5a344739b58ec8313639496, user_id=adbba7be39a64230bab160ce77ce7586, role_id=9a8de43206fa4a8fa4901488bf86f9f5)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:36.937757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b5bce-fdf4-4769-a51e-259a38c8f95a None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:36.972800 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 205/415] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:35:36 2022] PUT /identity/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/adbba7be39a64230bab160ce77ce7586/roles/9a8de43206fa4a8fa4901488bf86f9f5 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:35:37.006280 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc92a32-082f-42ae-b682-d0775449d5c2 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:37.032492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:37.051227 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f1c18-b4d1-49dd-a65d-e8d191661094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:37.051898 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f1c18-b4d1-49dd-a65d-e8d191661094 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:37.052225 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f1c18-b4d1-49dd-a65d-e8d191661094 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:37.052549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f1c18-b4d1-49dd-a65d-e8d191661094 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:37.064251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f1c18-b4d1-49dd-a65d-e8d191661094 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:37.065806 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f1c18-b4d1-49dd-a65d-e8d191661094 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:37.079666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-016f1c18-b4d1-49dd-a65d-e8d191661094 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:37.079666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:37.079666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:37.079666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:37.079666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:37.079666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:37.079666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:37.079666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:37.079666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:37.079666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:37.079666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:37.079666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:37.079666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:35:37.079666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:35:37.079666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:35:37.080678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:35:37.080678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:37.080678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:37.080678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:37.080678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:37.080678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:37.080678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:37.080678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:37.080678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:37.080678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:37.080678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:37.080678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:37.080678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:37.080678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:37.080678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:37.080678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:37.080678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:37.081525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:35:37.081525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:35:37.081525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:35:37.081525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:35:37.081525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:35:37.081525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:35:37.081525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:37.081525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:37.081525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 211/416] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:37.086798 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b5bce-fdf4-4769-a51e-259a38c8f95a None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:37.100741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:37.221671 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b5bce-fdf4-4769-a51e-259a38c8f95a None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:37.259777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b108d9-8516-42b9-878d-fa846eb7dc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:37.260507 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b108d9-8516-42b9-878d-fa846eb7dc5f None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:37.260791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b108d9-8516-42b9-878d-fa846eb7dc5f None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:37.261064 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b108d9-8516-42b9-878d-fa846eb7dc5f None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:37.261772 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b108d9-8516-42b9-878d-fa846eb7dc5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:37.262080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b108d9-8516-42b9-878d-fa846eb7dc5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:37.263609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b108d9-8516-42b9-878d-fa846eb7dc5f None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:37.279190 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 206/417] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:35:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:37.285092 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f1c18-b4d1-49dd-a65d-e8d191661094 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:37.408348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f1c18-b4d1-49dd-a65d-e8d191661094 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:37.455603 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f202047-e9e0-45c0-9fb6-145286d4ea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:37.456356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f202047-e9e0-45c0-9fb6-145286d4ea83 None admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:37.456639 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f202047-e9e0-45c0-9fb6-145286d4ea83 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:37.456968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f202047-e9e0-45c0-9fb6-145286d4ea83 None admin] PATH_INFO: `/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/fb39b6cc97a1456bb1312f49118c6e10/roles/9a8de43206fa4a8fa4901488bf86f9f5` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:37.480050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f202047-e9e0-45c0-9fb6-145286d4ea83 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab3a0e5fc5a344739b58ec8313639496, user_id=fb39b6cc97a1456bb1312f49118c6e10, role_id=9a8de43206fa4a8fa4901488bf86f9f5)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:37.481525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f202047-e9e0-45c0-9fb6-145286d4ea83 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:37.500900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 212/418] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:35:37 2022] PUT /identity/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/fb39b6cc97a1456bb1312f49118c6e10/roles/9a8de43206fa4a8fa4901488bf86f9f5 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:35:38.239747 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5673b49-4e3e-49c4-84e4-b8e9034470fb None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:38.240311 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5673b49-4e3e-49c4-84e4-b8e9034470fb None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:38.240311 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5673b49-4e3e-49c4-84e4-b8e9034470fb None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:38.241909 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 207/419] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:38.249343 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d0cae-85ec-4ab0-8c17-464e0d8ab3fa None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:38.249810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d0cae-85ec-4ab0-8c17-464e0d8ab3fa None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:38.249901 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d0cae-85ec-4ab0-8c17-464e0d8ab3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:38.278392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-2e0d0cae-85ec-4ab0-8c17-464e0d8ab3fa None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:38.279827 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-2e0d0cae-85ec-4ab0-8c17-464e0d8ab3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:38.303730 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:38.350106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb1edc-333b-407d-a1b6-67c2b251e7d3 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:38.350415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb1edc-333b-407d-a1b6-67c2b251e7d3 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:38.350656 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb1edc-333b-407d-a1b6-67c2b251e7d3 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:38.352222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 208/420] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:35:38.362194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01465c0a-b855-49e8-8652-5e0d92e5a999 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:38.362498 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01465c0a-b855-49e8-8652-5e0d92e5a999 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:38.362756 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01465c0a-b855-49e8-8652-5e0d92e5a999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:38.391021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-01465c0a-b855-49e8-8652-5e0d92e5a999 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:38.392264 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-01465c0a-b855-49e8-8652-5e0d92e5a999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:38.396281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 213/421] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:38.472803 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:38.486686 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 209/422] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:35:38.681831 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d0cae-85ec-4ab0-8c17-464e0d8ab3fa None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:38.814866 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d0cae-85ec-4ab0-8c17-464e0d8ab3fa None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:38.831007 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01465c0a-b855-49e8-8652-5e0d92e5a999 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:38.879504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d516ff1-1bf3-4c19-bcbf-fbf832acd4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:38.880172 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d516ff1-1bf3-4c19-bcbf-fbf832acd4db None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:38.880456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d516ff1-1bf3-4c19-bcbf-fbf832acd4db None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:38.880729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d516ff1-1bf3-4c19-bcbf-fbf832acd4db None admin] PATH_INFO: `/v3/roles/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:38.900336 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d516ff1-1bf3-4c19-bcbf-fbf832acd4db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:38.910596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-2d516ff1-1bf3-4c19-bcbf-fbf832acd4db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:38.910596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:38.910596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:38.910596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:38.910596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:38.910596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:38.910596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:38.910596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:38.910596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:38.910596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:38.910596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:38.910596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:38.910596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:38.910596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:38.910596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:38.910596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:38.911521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:38.911521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:38.911521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:38.911521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:38.911521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:38.911521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:38.911521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:38.911521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:38.911521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:38.911521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:38.911521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:38.911521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:38.911521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:38.911521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:38.911521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:38.911521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:38.912366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:38.912366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:38.912366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:38.912366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:38.912366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:38.912366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:38.912366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:38.912366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:38.912366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:38.912366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:38.912366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:38.912366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:38.912366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 214/423] 10.222.0.36 () {56 vars in 1102 bytes} [Sun Oct 30 00:35:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:38.921956 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d516ff1-1bf3-4c19-bcbf-fbf832acd4db None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:38.959474 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:39.013174 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01465c0a-b855-49e8-8652-5e0d92e5a999 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:39.076428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a8f7a-81a2-4445-aaf3-f33d1b3f9d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:39.078529 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a8f7a-81a2-4445-aaf3-f33d1b3f9d35 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:39.079225 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a8f7a-81a2-4445-aaf3-f33d1b3f9d35 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:39.080384 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a8f7a-81a2-4445-aaf3-f33d1b3f9d35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:39.100992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1a8f7a-81a2-4445-aaf3-f33d1b3f9d35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:39.106456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-ac1a8f7a-81a2-4445-aaf3-f33d1b3f9d35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:39.106456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:39.106456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:39.106456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:39.106456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:39.106456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:39.106456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:39.106456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:39.106456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:39.106456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:39.106456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:39.106456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:39.106456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:39.106456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:39.106456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:39.106456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:39.107366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:39.107366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:39.107366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:39.107366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:39.107366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:39.107366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:39.107366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:39.107366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:39.107366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:39.107366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:39.107366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:39.107366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:39.107366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:39.107366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:39.107366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:39.107366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:39.108244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:39.108244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:39.108244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:39.108244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:39.108244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:39.108244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:39.108244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:39.108244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:39.108244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:39.108244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:39.108244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:39.108244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:39.114509 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d516ff1-1bf3-4c19-bcbf-fbf832acd4db None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:39.123773 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 210/424] 10.222.0.36 () {56 vars in 1102 bytes} [Sun Oct 30 00:35:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:39.124954 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a8f7a-81a2-4445-aaf3-f33d1b3f9d35 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:39.182659 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ede5aa-1047-49d3-9436-3b182b077bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:39.183279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ede5aa-1047-49d3-9436-3b182b077bd5 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:39.183559 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ede5aa-1047-49d3-9436-3b182b077bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:39.183833 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ede5aa-1047-49d3-9436-3b182b077bd5 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:39.187551 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ede5aa-1047-49d3-9436-3b182b077bd5 None admin] RBAC: Adding query filter params (name=service) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:39.187871 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ede5aa-1047-49d3-9436-3b182b077bd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:39.189304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ede5aa-1047-49d3-9436-3b182b077bd5 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:39.209464 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 215/425] 10.222.0.36 () {56 vars in 1103 bytes} [Sun Oct 30 00:35:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:39.215801 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ede5aa-1047-49d3-9436-3b182b077bd5 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:39.238435 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:39.330235 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a8f7a-81a2-4445-aaf3-f33d1b3f9d35 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:39.362405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:39.405795 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe108d-f933-49ae-83cf-1fb39783eea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:39.406473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe108d-f933-49ae-83cf-1fb39783eea5 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:39.406759 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe108d-f933-49ae-83cf-1fb39783eea5 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:39.407031 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe108d-f933-49ae-83cf-1fb39783eea5 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:39.407708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe108d-f933-49ae-83cf-1fb39783eea5 None admin] RBAC: Adding query filter params (name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:39.408009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe108d-f933-49ae-83cf-1fb39783eea5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:39.409435 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe108d-f933-49ae-83cf-1fb39783eea5 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:39.413029 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ede5aa-1047-49d3-9436-3b182b077bd5 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:39.423366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 211/426] 10.222.0.36 () {56 vars in 1103 bytes} [Sun Oct 30 00:35:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:39.431583 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe108d-f933-49ae-83cf-1fb39783eea5 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:39.457155 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14292832-d147-460e-a0d4-8dfb79fb557f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:39.457798 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14292832-d147-460e-a0d4-8dfb79fb557f None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:39.458078 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14292832-d147-460e-a0d4-8dfb79fb557f None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:39.458351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14292832-d147-460e-a0d4-8dfb79fb557f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:39.471336 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14292832-d147-460e-a0d4-8dfb79fb557f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:39.472774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14292832-d147-460e-a0d4-8dfb79fb557f None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:39.489438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-14292832-d147-460e-a0d4-8dfb79fb557f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:39.489438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:39.489438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:39.489438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:39.489438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:39.489438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:39.489438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:39.489438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:39.489438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:39.489438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:39.489438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:39.489438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:39.489438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:39.489438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:39.489438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:39.489438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:39.490355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:39.490355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:39.490355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:39.490355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:39.490355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:39.490355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:39.490355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:39.490355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:39.490355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:39.490355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:39.490355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:39.490355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:39.490355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:39.490355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:39.490355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:39.490355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:39.490355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:39.491210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:39.491210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:39.491210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:39.491210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 216/427] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:35:39.498011 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14292832-d147-460e-a0d4-8dfb79fb557f None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:39.653612 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe108d-f933-49ae-83cf-1fb39783eea5 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:39.687608 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:39.694880 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14292832-d147-460e-a0d4-8dfb79fb557f None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:39.721708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c17af-7ed1-4e88-96eb-27128f98eda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:39.722297 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c17af-7ed1-4e88-96eb-27128f98eda8 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:39.722636 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c17af-7ed1-4e88-96eb-27128f98eda8 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:39.722833 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c17af-7ed1-4e88-96eb-27128f98eda8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:39.740483 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:39.751297 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c17af-7ed1-4e88-96eb-27128f98eda8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:39.754691 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c17af-7ed1-4e88-96eb-27128f98eda8 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:39.756318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f9704-9252-4f36-86db-1272ab1af137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:39.756943 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f9704-9252-4f36-86db-1272ab1af137 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:39.757224 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f9704-9252-4f36-86db-1272ab1af137 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:39.757496 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f9704-9252-4f36-86db-1272ab1af137 None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:39.758151 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f9704-9252-4f36-86db-1272ab1af137 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:39.758451 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f9704-9252-4f36-86db-1272ab1af137 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:39.759869 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f9704-9252-4f36-86db-1272ab1af137 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:39.779685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-e79c17af-7ed1-4e88-96eb-27128f98eda8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:39.779685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:39.779685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:39.779685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:39.779685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:39.779685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:39.779685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:39.779685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:39.779685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:39.779685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:39.779685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:39.779685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:39.779685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:39.779685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:39.779685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:39.779685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:39.780726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:39.780726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:39.780726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:39.780726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:39.780726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:39.780726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:39.780726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:39.780726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:39.780726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:39.780726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:39.780726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:39.780726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:39.780726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:39.780726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:39.780726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:39.780726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:39.780726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:39.781593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:39.781593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:39.781593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:39.781764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 217/428] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:39.781838 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 212/429] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:39.781903 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175d90e-97cf-42db-b3d9-abb56cc55337 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:39.782420 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175d90e-97cf-42db-b3d9-abb56cc55337 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:39.782420 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175d90e-97cf-42db-b3d9-abb56cc55337 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:39.783498 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 218/430] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:39.787125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c17af-7ed1-4e88-96eb-27128f98eda8 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:39.788638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175d90e-97cf-42db-b3d9-abb56cc55337 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:39.901280 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175d90e-97cf-42db-b3d9-abb56cc55337 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:39.916364 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c17af-7ed1-4e88-96eb-27128f98eda8 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:39.943017 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073e4ac-6359-4fda-a693-fc6e74284442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:39.943650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073e4ac-6359-4fda-a693-fc6e74284442 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:39.943934 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073e4ac-6359-4fda-a693-fc6e74284442 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:39.944243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073e4ac-6359-4fda-a693-fc6e74284442 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:39.952812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073e4ac-6359-4fda-a693-fc6e74284442 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:39.953820 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f5f99-c754-448c-8f9b-732cf2454eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:39.954288 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073e4ac-6359-4fda-a693-fc6e74284442 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:39.954468 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f5f99-c754-448c-8f9b-732cf2454eca None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:39.954776 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f5f99-c754-448c-8f9b-732cf2454eca None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:39.955066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f5f99-c754-448c-8f9b-732cf2454eca None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:39.955711 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f5f99-c754-448c-8f9b-732cf2454eca None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:39.956017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f5f99-c754-448c-8f9b-732cf2454eca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:39.957409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f5f99-c754-448c-8f9b-732cf2454eca None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:39.966620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-3073e4ac-6359-4fda-a693-fc6e74284442 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 30 00:35:39.966620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:39.966620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:39.966620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:39.966620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:39.966620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:39.966620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:39.966620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:39.966620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:39.966620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:39.966620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:39.966620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:39.966620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:35:39.966620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:35:39.966620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:35:39.966620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:35:39.967710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:39.967710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:39.967710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:35:39.967710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:39.967710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:35:39.967710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:39.967710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:39.967710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:39.967710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:39.967710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:39.967710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:39.967710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:39.967710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:39.967710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:39.967710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:39.967710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:39.967710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:39.967710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:39.968692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:39.968692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:39.968692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:35:39.968692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:35:39.968692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:35:39.968692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:35:39.968692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:35:39.968692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:35:39.968692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 30 00:35:39.968692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:39.968692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 219/431] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:35:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:39.972765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 213/432] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:39.973509 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced9a16-5f6c-4656-a46e-9db9e2221b7e None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:39.973796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced9a16-5f6c-4656-a46e-9db9e2221b7e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:39.974079 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced9a16-5f6c-4656-a46e-9db9e2221b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:39.974602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e7d16-590f-43f7-a8d2-de7e67e52406 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:39.975013 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e7d16-590f-43f7-a8d2-de7e67e52406 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:39.975409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e7d16-590f-43f7-a8d2-de7e67e52406 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:39.981501 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 214/433] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:39.981501 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e7d16-590f-43f7-a8d2-de7e67e52406 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:39.993216 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-bced9a16-5f6c-4656-a46e-9db9e2221b7e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:39.994519 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-bced9a16-5f6c-4656-a46e-9db9e2221b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:40.017536 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:40.053535 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:40.078656 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 220/434] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:40.079515 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced9a16-5f6c-4656-a46e-9db9e2221b7e None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:40.122715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e7d16-590f-43f7-a8d2-de7e67e52406 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:40.163820 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04221f63-7d58-4a61-9011-e77058099f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:40.167334 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04221f63-7d58-4a61-9011-e77058099f0b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:40.167857 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04221f63-7d58-4a61-9011-e77058099f0b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:40.168492 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04221f63-7d58-4a61-9011-e77058099f0b None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:40.169564 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04221f63-7d58-4a61-9011-e77058099f0b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:40.170128 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04221f63-7d58-4a61-9011-e77058099f0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:40.171924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04221f63-7d58-4a61-9011-e77058099f0b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:40.202415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 215/435] 10.222.0.36 () {56 vars in 1135 bytes} [Sun Oct 30 00:35:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:40.202415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811cc22-83be-4819-a13e-313ce91a62f7 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:40.202415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811cc22-83be-4819-a13e-313ce91a62f7 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:40.202415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811cc22-83be-4819-a13e-313ce91a62f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:40.226649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced9a16-5f6c-4656-a46e-9db9e2221b7e None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:40.227175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-8811cc22-83be-4819-a13e-313ce91a62f7 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:40.229163 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-8811cc22-83be-4819-a13e-313ce91a62f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:40.275467 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d440b-f1d7-4feb-94d9-b24917754f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:40.276369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d440b-f1d7-4feb-94d9-b24917754f3c None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:40.276459 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d440b-f1d7-4feb-94d9-b24917754f3c None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:40.276827 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d440b-f1d7-4feb-94d9-b24917754f3c None admin] PATH_INFO: `/v3/users/glance` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:40.286399 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d440b-f1d7-4feb-94d9-b24917754f3c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:40.288687 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d440b-f1d7-4feb-94d9-b24917754f3c None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:40.305542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-e16d440b-f1d7-4feb-94d9-b24917754f3c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 30 00:35:40.305542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:40.305542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:40.305542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:40.305542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:40.305542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:40.305542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:40.305542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:40.305542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:40.305542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:40.305542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:40.305542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:40.305542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:35:40.305542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:35:40.305542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:35:40.305542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:35:40.307191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:40.307191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:40.307191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:35:40.307191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:40.307191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:35:40.307191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:40.307191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:40.307191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:40.307191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:40.307191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:40.307191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:40.307191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:40.307191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:40.307191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:40.307191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:40.307191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:40.307191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:40.307191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:40.308348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:40.308348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:40.308348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:35:40.308348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:35:40.308348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:35:40.308348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:35:40.308348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:35:40.308348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:35:40.308348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 30 00:35:40.308348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:40.308348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 221/436] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:40.308348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d440b-f1d7-4feb-94d9-b24917754f3c None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:40.324673 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 216/437] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:40.326253 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2a422-b4cf-4b6b-ab06-1164d6ffee82 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:40.326709 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2a422-b4cf-4b6b-ab06-1164d6ffee82 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:40.327120 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2a422-b4cf-4b6b-ab06-1164d6ffee82 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:40.328610 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 217/438] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:40.329456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2a422-b4cf-4b6b-ab06-1164d6ffee82 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:40.352396 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:40.363212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:40.444822 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d440b-f1d7-4feb-94d9-b24917754f3c None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:40.465310 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2a422-b4cf-4b6b-ab06-1164d6ffee82 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:40.496451 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f4202-f403-46c7-9648-a14dbb6afc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:40.496995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f4202-f403-46c7-9648-a14dbb6afc32 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:40.497274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f4202-f403-46c7-9648-a14dbb6afc32 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:40.497549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f4202-f403-46c7-9648-a14dbb6afc32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:40.507786 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f8619-fb96-4e95-9f43-18e43258d0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:40.508422 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f8619-fb96-4e95-9f43-18e43258d0bf None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:40.508699 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f8619-fb96-4e95-9f43-18e43258d0bf None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:40.508968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f8619-fb96-4e95-9f43-18e43258d0bf None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:40.509922 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f8619-fb96-4e95-9f43-18e43258d0bf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:40.510229 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f8619-fb96-4e95-9f43-18e43258d0bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:40.510405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f4202-f403-46c7-9648-a14dbb6afc32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:40.511659 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f8619-fb96-4e95-9f43-18e43258d0bf None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:40.512336 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f4202-f403-46c7-9648-a14dbb6afc32 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:40.530550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-f78f4202-f403-46c7-9648-a14dbb6afc32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:40.530550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:40.530550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:40.530550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:40.530550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:40.530550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:40.530550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:40.530550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:40.530550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:40.530550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:40.530550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:40.530550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:40.530550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:40.530550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:40.530550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:40.530550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:40.531537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:40.531537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:40.531537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:40.531537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:40.531537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:40.531537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:40.531537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:40.531537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:40.531537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:40.531537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:40.531537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:40.531537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:40.531537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:40.531537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:40.531537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:40.531537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:40.531537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:40.532914 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:40.532914 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:40.532914 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:40.533309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 222/439] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:40.538376 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cc5a8-97b0-45ae-814b-13f6d5277907 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:40.538376 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cc5a8-97b0-45ae-814b-13f6d5277907 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:40.538580 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 218/440] 10.222.0.36 () {56 vars in 1137 bytes} [Sun Oct 30 00:35:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:40.538676 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cc5a8-97b0-45ae-814b-13f6d5277907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:40.538787 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f8619-fb96-4e95-9f43-18e43258d0bf None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:40.562945 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-143cc5a8-97b0-45ae-814b-13f6d5277907 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:40.564233 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-143cc5a8-97b0-45ae-814b-13f6d5277907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:40.628061 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:40.649935 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 223/441] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:40.650447 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cc5a8-97b0-45ae-814b-13f6d5277907 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:40.656708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f8619-fb96-4e95-9f43-18e43258d0bf None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:40.683829 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:40.694233 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc53b71-41bc-42bb-8fec-a6c0d4793b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:40.694873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc53b71-41bc-42bb-8fec-a6c0d4793b1b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:40.695156 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc53b71-41bc-42bb-8fec-a6c0d4793b1b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:40.695434 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc53b71-41bc-42bb-8fec-a6c0d4793b1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:40.706574 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc53b71-41bc-42bb-8fec-a6c0d4793b1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:40.711414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-8dc53b71-41bc-42bb-8fec-a6c0d4793b1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:40.711414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:40.711414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:40.711414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:40.711414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:40.711414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:40.711414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:40.711414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:40.711414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:40.711414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:40.711414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:40.711414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:40.711414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:40.711414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:40.711414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:40.711414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:40.713248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:40.713248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:40.713248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:40.713248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:40.713248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:40.713248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:40.713248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:40.713248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:40.713248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:40.713248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:40.713248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:40.713248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:40.713248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:40.713248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:40.713248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:40.713248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:40.714216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:40.714216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:40.714216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:40.714216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:40.714216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:40.714216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:40.714216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:40.714216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:40.714216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:40.714216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:40.714216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:40.714216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:40.714216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 219/442] 10.222.0.36 () {56 vars in 1102 bytes} [Sun Oct 30 00:35:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:40.714216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc53b71-41bc-42bb-8fec-a6c0d4793b1b None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:40.782014 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cc5a8-97b0-45ae-814b-13f6d5277907 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:40.819996 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce321ed-8db5-446f-9774-7b757884cb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:40.822347 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce321ed-8db5-446f-9774-7b757884cb40 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:40.822930 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce321ed-8db5-446f-9774-7b757884cb40 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:40.823205 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce321ed-8db5-446f-9774-7b757884cb40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:40.832360 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc53b71-41bc-42bb-8fec-a6c0d4793b1b None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:40.877401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2df760-c056-4b34-bfc3-016072c5aead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:40.878464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2df760-c056-4b34-bfc3-016072c5aead None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:40.878946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2df760-c056-4b34-bfc3-016072c5aead None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:40.879447 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2df760-c056-4b34-bfc3-016072c5aead None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:40.880686 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2df760-c056-4b34-bfc3-016072c5aead None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:40.882821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2df760-c056-4b34-bfc3-016072c5aead None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:40.883970 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2df760-c056-4b34-bfc3-016072c5aead None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:40.886964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:40.902547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 220/443] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:40.903399 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2df760-c056-4b34-bfc3-016072c5aead None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:40.914402 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce321ed-8db5-446f-9774-7b757884cb40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:40.917582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce321ed-8db5-446f-9774-7b757884cb40 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:40.946499 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:41.016773 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2df760-c056-4b34-bfc3-016072c5aead None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:41.038404 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 224/444] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:35:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:35:41.039645 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce321ed-8db5-446f-9774-7b757884cb40 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:41.052610 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3dfcd-621f-4d8f-adef-ea1eb073d1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:41.053253 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3dfcd-621f-4d8f-adef-ea1eb073d1b2 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:41.053535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3dfcd-621f-4d8f-adef-ea1eb073d1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:41.053811 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3dfcd-621f-4d8f-adef-ea1eb073d1b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:41.065810 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b3dfcd-621f-4d8f-adef-ea1eb073d1b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:41.071092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-88b3dfcd-621f-4d8f-adef-ea1eb073d1b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:41.071092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:41.071092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:41.071092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:41.071092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:41.071092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:41.071092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:41.071092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:41.071092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:41.071092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:41.071092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:41.071092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:41.071092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:41.071092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:41.071092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:41.071092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:41.072791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:41.072791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:41.072791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:41.072791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:41.072791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:41.072791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:41.072791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:41.072791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:41.072791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:41.072791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:41.072791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:41.072791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:41.072791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:41.072791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:41.072791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:41.072791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:41.073629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:41.073629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:41.073629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:41.073629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:41.073629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:41.073629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:41.073629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:41.073629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:41.073629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:41.073629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:41.073629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:41.073629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:41.076231 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 221/445] 10.222.0.36 () {56 vars in 1102 bytes} [Sun Oct 30 00:35:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:41.077330 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3dfcd-621f-4d8f-adef-ea1eb073d1b2 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:41.136828 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:41.155379 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce321ed-8db5-446f-9774-7b757884cb40 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:41.176340 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:41.184053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3dfcd-621f-4d8f-adef-ea1eb073d1b2 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:41.191291 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c3ffe-96d8-44e4-8054-d129217a6201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:41.191893 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c3ffe-96d8-44e4-8054-d129217a6201 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:41.192198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c3ffe-96d8-44e4-8054-d129217a6201 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:41.192474 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c3ffe-96d8-44e4-8054-d129217a6201 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:41.193130 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c3ffe-96d8-44e4-8054-d129217a6201 None admin] RBAC: Adding query filter params (name=service) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:41.193427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c3ffe-96d8-44e4-8054-d129217a6201 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:41.194804 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c3ffe-96d8-44e4-8054-d129217a6201 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:41.203299 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 225/446] 10.222.0.36 () {56 vars in 1103 bytes} [Sun Oct 30 00:35:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:41.204132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c3ffe-96d8-44e4-8054-d129217a6201 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:41.219020 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1992ed-f82e-4bc6-8353-964381cc6915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:41.219654 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1992ed-f82e-4bc6-8353-964381cc6915 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:41.219936 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1992ed-f82e-4bc6-8353-964381cc6915 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:41.220241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1992ed-f82e-4bc6-8353-964381cc6915 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:41.226973 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1992ed-f82e-4bc6-8353-964381cc6915 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:41.228674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1992ed-f82e-4bc6-8353-964381cc6915 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:41.239921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-ab1992ed-f82e-4bc6-8353-964381cc6915 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:41.239921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:41.239921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:41.239921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:41.239921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:41.239921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:41.239921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:41.239921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:41.239921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:41.239921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:41.239921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:41.239921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:41.239921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:35:41.239921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:35:41.239921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:35:41.240963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:35:41.240963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:41.240963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:41.240963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:41.240963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:41.240963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:41.240963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:41.240963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:41.240963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:41.240963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:41.240963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:41.240963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:41.240963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:41.240963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:41.240963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:41.240963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:41.240963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:41.241873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:35:41.241873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:35:41.241873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:35:41.241873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:35:41.241873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:35:41.241873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:35:41.241873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:41.241873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:41.241873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 222/447] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:41.242335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1992ed-f82e-4bc6-8353-964381cc6915 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:41.305418 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c3ffe-96d8-44e4-8054-d129217a6201 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:41.340010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e885b02-a3c9-425f-befc-ea9ef0361c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:41.340670 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e885b02-a3c9-425f-befc-ea9ef0361c7a None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:41.340946 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e885b02-a3c9-425f-befc-ea9ef0361c7a None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:41.341221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e885b02-a3c9-425f-befc-ea9ef0361c7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:41.346007 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1992ed-f82e-4bc6-8353-964381cc6915 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:41.351297 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e885b02-a3c9-425f-befc-ea9ef0361c7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:41.356605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-7e885b02-a3c9-425f-befc-ea9ef0361c7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:41.356605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:41.356605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:41.356605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:41.356605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:41.356605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:41.356605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:41.356605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:41.356605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:41.356605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:41.356605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:41.356605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:41.356605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:41.356605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:41.356605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:41.356605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:41.357547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:41.357547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:41.357547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:41.357547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:41.357547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:41.357547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:41.357547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:41.357547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:41.357547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:41.357547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:41.357547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:41.357547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:41.357547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:41.357547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:41.357547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:41.357547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:41.358437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:41.358437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:41.358437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:41.358437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:41.358437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:41.358437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:41.358437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:41.358437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:41.358437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:41.358437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:41.358437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:41.358437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:41.358437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 226/448] 10.222.0.36 () {56 vars in 1102 bytes} [Sun Oct 30 00:35:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:41.359159 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e885b02-a3c9-425f-befc-ea9ef0361c7a None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:41.364754 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:41.386669 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056e2be-b6ee-4b88-8d90-d17f7a009562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:41.387302 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056e2be-b6ee-4b88-8d90-d17f7a009562 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:41.387580 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056e2be-b6ee-4b88-8d90-d17f7a009562 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:41.387854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056e2be-b6ee-4b88-8d90-d17f7a009562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:41.398317 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056e2be-b6ee-4b88-8d90-d17f7a009562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:41.399792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056e2be-b6ee-4b88-8d90-d17f7a009562 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:41.404981 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:41.415471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-c056e2be-b6ee-4b88-8d90-d17f7a009562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:41.415471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:41.415471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:41.415471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:41.415471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:41.415471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:41.415471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:41.415471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:41.415471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:41.415471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:41.415471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:41.415471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:41.415471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:41.415471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:41.415471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:41.415471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:41.416582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:41.416582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:41.416582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:41.416582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:41.416582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:41.416582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:41.416582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:41.416582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:41.416582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:41.416582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:41.416582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:41.416582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:41.416582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:41.416582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:41.416582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:41.416582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:41.416582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:41.417448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:41.417448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:41.417448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:41.417448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 223/449] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:41.417448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056e2be-b6ee-4b88-8d90-d17f7a009562 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:41.466208 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e885b02-a3c9-425f-befc-ea9ef0361c7a None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:41.499873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2866263-8512-4165-8950-829d1c9d2702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:41.500559 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2866263-8512-4165-8950-829d1c9d2702 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:41.500835 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2866263-8512-4165-8950-829d1c9d2702 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:41.501110 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2866263-8512-4165-8950-829d1c9d2702 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:41.501776 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2866263-8512-4165-8950-829d1c9d2702 None admin] RBAC: Adding query filter params (name=service) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:41.502077 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2866263-8512-4165-8950-829d1c9d2702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:41.503458 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2866263-8512-4165-8950-829d1c9d2702 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:41.512129 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 227/450] 10.222.0.36 () {56 vars in 1103 bytes} [Sun Oct 30 00:35:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:41.512883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2866263-8512-4165-8950-829d1c9d2702 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:41.519255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056e2be-b6ee-4b88-8d90-d17f7a009562 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:41.552458 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc739a8-5251-4dbc-8b90-f3c23dae7cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:41.553090 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc739a8-5251-4dbc-8b90-f3c23dae7cb8 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:41.553372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc739a8-5251-4dbc-8b90-f3c23dae7cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:41.553647 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc739a8-5251-4dbc-8b90-f3c23dae7cb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:41.564981 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc739a8-5251-4dbc-8b90-f3c23dae7cb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:41.566374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc739a8-5251-4dbc-8b90-f3c23dae7cb8 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:41.581066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-4cc739a8-5251-4dbc-8b90-f3c23dae7cb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:41.581066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:41.581066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:41.581066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:41.581066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:41.581066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:41.581066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:41.581066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:41.581066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:41.581066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:41.581066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:41.581066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:41.581066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:41.581066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:41.581066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:41.581066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:41.582183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:41.582183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:41.582183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:41.582183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:41.582183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:41.582183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:41.582183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:41.582183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:41.582183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:41.582183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:41.582183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:41.582183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:41.582183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:41.582183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:41.582183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:41.582183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:41.582183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:41.583126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:41.583126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:41.583126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:41.583126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 224/451] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:41.583126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc739a8-5251-4dbc-8b90-f3c23dae7cb8 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:41.602551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:41.616692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2866263-8512-4165-8950-829d1c9d2702 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:41.620420 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:41.653429 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e03e00-6176-42a1-b524-af612abc6fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:41.654038 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e03e00-6176-42a1-b524-af612abc6fe9 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:41.654324 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e03e00-6176-42a1-b524-af612abc6fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:41.654593 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e03e00-6176-42a1-b524-af612abc6fe9 None admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:41.655300 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e03e00-6176-42a1-b524-af612abc6fe9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:41.655602 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e03e00-6176-42a1-b524-af612abc6fe9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:41.657069 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e03e00-6176-42a1-b524-af612abc6fe9 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:41.669402 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 228/452] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:35:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:41.670203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e03e00-6176-42a1-b524-af612abc6fe9 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:41.688837 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc739a8-5251-4dbc-8b90-f3c23dae7cb8 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:41.723922 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd011d-f619-4bb2-81a0-65c4fe104ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:41.724562 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd011d-f619-4bb2-81a0-65c4fe104ef8 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:41.725000 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd011d-f619-4bb2-81a0-65c4fe104ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:41.725280 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd011d-f619-4bb2-81a0-65c4fe104ef8 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:41.726294 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd011d-f619-4bb2-81a0-65c4fe104ef8 None admin] RBAC: Adding query filter params (name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:41.726597 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd011d-f619-4bb2-81a0-65c4fe104ef8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:41.727965 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd011d-f619-4bb2-81a0-65c4fe104ef8 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:41.736305 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 225/453] 10.222.0.36 () {56 vars in 1103 bytes} [Sun Oct 30 00:35:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:41.737176 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd011d-f619-4bb2-81a0-65c4fe104ef8 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:41.768988 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e03e00-6176-42a1-b524-af612abc6fe9 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:41.802851 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ce9ce-2143-452d-87f0-86b416ebad6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:41.803453 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ce9ce-2143-452d-87f0-86b416ebad6e None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:41.803733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ce9ce-2143-452d-87f0-86b416ebad6e None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:41.804005 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ce9ce-2143-452d-87f0-86b416ebad6e None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:41.804696 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ce9ce-2143-452d-87f0-86b416ebad6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:41.804997 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ce9ce-2143-452d-87f0-86b416ebad6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:41.806925 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ce9ce-2143-452d-87f0-86b416ebad6e None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:41.818309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 229/454] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:41.819040 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ce9ce-2143-452d-87f0-86b416ebad6e None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:41.825826 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:41.844741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:41.846701 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd011d-f619-4bb2-81a0-65c4fe104ef8 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:41.880962 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b36732-9ce9-4938-88a5-b659db29bb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:41.881603 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b36732-9ce9-4938-88a5-b659db29bb05 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:41.881883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b36732-9ce9-4938-88a5-b659db29bb05 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:41.882157 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b36732-9ce9-4938-88a5-b659db29bb05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:41.894113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b36732-9ce9-4938-88a5-b659db29bb05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:41.895570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b36732-9ce9-4938-88a5-b659db29bb05 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:41.911166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-f8b36732-9ce9-4938-88a5-b659db29bb05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:41.911166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:41.911166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:41.911166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:41.911166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:41.911166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:41.911166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:41.911166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:41.911166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:41.911166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:41.911166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:41.911166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:41.911166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:41.911166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:41.911166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:41.911166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:41.912263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:41.912263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:41.912263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:41.912263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:41.912263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:41.912263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:41.912263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:41.912263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:41.912263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:41.912263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:41.912263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:41.912263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:41.912263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:41.912263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:41.912263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:41.912263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:41.912263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:41.913285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:41.913285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:41.913285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:41.913285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 226/455] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:41.914606 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b36732-9ce9-4938-88a5-b659db29bb05 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:41.923423 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ce9ce-2143-452d-87f0-86b416ebad6e None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:41.958117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031967f-0b6a-49f0-8b18-361c4b33095c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:41.958723 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031967f-0b6a-49f0-8b18-361c4b33095c None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:41.959000 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031967f-0b6a-49f0-8b18-361c4b33095c None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:41.959269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031967f-0b6a-49f0-8b18-361c4b33095c None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:41.959912 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031967f-0b6a-49f0-8b18-361c4b33095c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:41.960242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031967f-0b6a-49f0-8b18-361c4b33095c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:41.961612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031967f-0b6a-49f0-8b18-361c4b33095c None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:41.973615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 230/456] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:41.974424 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031967f-0b6a-49f0-8b18-361c4b33095c None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:42.021025 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b36732-9ce9-4938-88a5-b659db29bb05 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:42.050392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:42.055873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a918df0-da94-43f5-a7df-345ff141e913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:42.056595 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a918df0-da94-43f5-a7df-345ff141e913 None admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:42.056869 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a918df0-da94-43f5-a7df-345ff141e913 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:42.057145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a918df0-da94-43f5-a7df-345ff141e913 None admin] PATH_INFO: `/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/9f4c779b8422432fb06cdf996841b9b5/roles/9a8de43206fa4a8fa4901488bf86f9f5` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:42.057504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:42.072531 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a918df0-da94-43f5-a7df-345ff141e913 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab3a0e5fc5a344739b58ec8313639496, user_id=9f4c779b8422432fb06cdf996841b9b5, role_id=9a8de43206fa4a8fa4901488bf86f9f5)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:42.074051 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a918df0-da94-43f5-a7df-345ff141e913 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:42.075680 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031967f-0b6a-49f0-8b18-361c4b33095c None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:42.086520 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 227/457] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:35:41 2022] PUT /identity/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/9f4c779b8422432fb06cdf996841b9b5/roles/9a8de43206fa4a8fa4901488bf86f9f5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:35:42.087347 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a918df0-da94-43f5-a7df-345ff141e913 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:42.129921 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b79eae-15db-469d-8d85-c93f83ee78a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:42.130577 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b79eae-15db-469d-8d85-c93f83ee78a5 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:42.130861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b79eae-15db-469d-8d85-c93f83ee78a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:42.131132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b79eae-15db-469d-8d85-c93f83ee78a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:42.142510 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b79eae-15db-469d-8d85-c93f83ee78a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:42.144041 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b79eae-15db-469d-8d85-c93f83ee78a5 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:42.160815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-34b79eae-15db-469d-8d85-c93f83ee78a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:42.160815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:42.160815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:42.160815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:42.160815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:42.160815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:42.160815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:42.160815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:42.160815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:42.160815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:42.160815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:42.160815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:42.160815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:42.160815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:42.160815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:42.160815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:42.161962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:42.161962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:42.161962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:42.161962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:42.161962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:42.161962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:42.161962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:42.161962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:42.161962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:42.161962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:42.161962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:42.161962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:42.161962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:42.161962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:42.161962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:42.161962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:42.161962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:42.162855 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:42.162855 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:42.162855 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:42.162855 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 231/458] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:42.163087 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b79eae-15db-469d-8d85-c93f83ee78a5 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:42.213565 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a918df0-da94-43f5-a7df-345ff141e913 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:42.253629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab95fc8-6a2e-4e70-ba22-0493bba83912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:42.254275 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab95fc8-6a2e-4e70-ba22-0493bba83912 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:42.254549 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab95fc8-6a2e-4e70-ba22-0493bba83912 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:42.254837 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab95fc8-6a2e-4e70-ba22-0493bba83912 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:42.262701 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab95fc8-6a2e-4e70-ba22-0493bba83912 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:42.264717 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab95fc8-6a2e-4e70-ba22-0493bba83912 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:42.278856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b79eae-15db-469d-8d85-c93f83ee78a5 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:42.279874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-0ab95fc8-6a2e-4e70-ba22-0493bba83912 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:42.279874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:42.279874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:42.279874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:42.279874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:42.279874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:42.279874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:42.279874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:42.279874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:42.279874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:42.279874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:42.279874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:42.279874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:35:42.279874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:35:42.279874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:35:42.280898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:35:42.280898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:42.280898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:42.280898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:42.280898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:42.280898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:42.280898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:42.280898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:42.280898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:42.280898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:42.280898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:42.280898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:42.280898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:42.280898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:42.280898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:42.280898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:42.280898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:42.281954 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:35:42.281954 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:35:42.281954 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:35:42.281954 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:35:42.281954 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:35:42.281954 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:35:42.281954 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:42.281954 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:42.281954 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 228/459] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:42.282523 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab95fc8-6a2e-4e70-ba22-0493bba83912 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:42.320423 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f87b2-94d0-410c-9d9d-c2156498165e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:42.321038 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f87b2-94d0-410c-9d9d-c2156498165e None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:42.321321 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f87b2-94d0-410c-9d9d-c2156498165e None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:42.321595 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f87b2-94d0-410c-9d9d-c2156498165e None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:42.322288 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f87b2-94d0-410c-9d9d-c2156498165e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:42.322644 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f87b2-94d0-410c-9d9d-c2156498165e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:42.324174 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f87b2-94d0-410c-9d9d-c2156498165e None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:42.326860 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:42.330254 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:42.344129 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 232/460] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:42.344129 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f87b2-94d0-410c-9d9d-c2156498165e None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:42.405583 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab95fc8-6a2e-4e70-ba22-0493bba83912 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:42.441521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c1f14-8943-4fe8-afdb-7551856c98c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:42.442132 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c1f14-8943-4fe8-afdb-7551856c98c6 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:42.442411 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c1f14-8943-4fe8-afdb-7551856c98c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:42.442691 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c1f14-8943-4fe8-afdb-7551856c98c6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:42.450753 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c1f14-8943-4fe8-afdb-7551856c98c6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:42.452279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c1f14-8943-4fe8-afdb-7551856c98c6 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:42.455837 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f87b2-94d0-410c-9d9d-c2156498165e None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:42.465241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-8f7c1f14-8943-4fe8-afdb-7551856c98c6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:35:42.465241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:42.465241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:42.465241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:42.465241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:42.465241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:42.465241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:42.465241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:42.465241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:42.465241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:42.465241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:42.465241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:42.465241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:35:42.465241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:35:42.465241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:35:42.465241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:35:42.466230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:42.466230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:42.466230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:35:42.466230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:42.466230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:35:42.466230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:42.466230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:42.466230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:42.466230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:42.466230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:42.466230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:42.466230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:42.466230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:42.466230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:42.466230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:42.466230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:42.466230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:42.466230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:42.467145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:42.467145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:42.467145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:35:42.467145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:35:42.467145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:35:42.467145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:35:42.467145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:35:42.467145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:35:42.467145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:35:42.467145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:42.471022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 229/461] 10.222.0.36 () {56 vars in 1093 bytes} [Sun Oct 30 00:35:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:42.471883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c1f14-8943-4fe8-afdb-7551856c98c6 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:42.496389 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e34f4-3a76-4d4e-8457-182e2b507ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:42.497118 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e34f4-3a76-4d4e-8457-182e2b507ac0 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:42.497402 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e34f4-3a76-4d4e-8457-182e2b507ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:42.497668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e34f4-3a76-4d4e-8457-182e2b507ac0 None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:42.498301 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e34f4-3a76-4d4e-8457-182e2b507ac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:42.498636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e34f4-3a76-4d4e-8457-182e2b507ac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:42.500032 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e34f4-3a76-4d4e-8457-182e2b507ac0 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:42.516531 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 233/462] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:42.516531 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e34f4-3a76-4d4e-8457-182e2b507ac0 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:42.577068 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:42.584239 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c1f14-8943-4fe8-afdb-7551856c98c6 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:42.585895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:42.624734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e34f4-3a76-4d4e-8457-182e2b507ac0 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:42.626534 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f97b81-2425-41d5-86bf-23baf695d491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:42.627153 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f97b81-2425-41d5-86bf-23baf695d491 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:42.627479 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f97b81-2425-41d5-86bf-23baf695d491 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:42.627752 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f97b81-2425-41d5-86bf-23baf695d491 None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:42.628496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f97b81-2425-41d5-86bf-23baf695d491 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:42.628799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f97b81-2425-41d5-86bf-23baf695d491 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:42.630232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f97b81-2425-41d5-86bf-23baf695d491 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:42.648363 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 230/463] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:35:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:42.648363 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f97b81-2425-41d5-86bf-23baf695d491 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:42.667707 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce2313-0286-49e4-9672-3adc3072aad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:42.668362 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce2313-0286-49e4-9672-3adc3072aad7 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:42.668640 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce2313-0286-49e4-9672-3adc3072aad7 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:42.668907 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce2313-0286-49e4-9672-3adc3072aad7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:42.676478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ce2313-0286-49e4-9672-3adc3072aad7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:42.677936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ce2313-0286-49e4-9672-3adc3072aad7 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:42.689768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-44ce2313-0286-49e4-9672-3adc3072aad7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 30 00:35:42.689768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:42.689768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:42.689768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:42.689768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:42.689768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:42.689768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:42.689768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:42.689768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:42.689768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:42.689768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:42.689768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:42.689768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:35:42.689768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:35:42.689768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:35:42.689768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:35:42.690753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:42.690753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:42.690753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:35:42.690753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:42.690753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:35:42.690753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:42.690753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:42.690753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:42.690753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:42.690753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:42.690753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:42.690753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:42.690753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:42.690753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:42.690753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:42.690753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:42.690753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:42.690753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:42.691654 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:42.691654 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:42.691654 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:35:42.691654 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:35:42.691654 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:35:42.691654 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:35:42.691654 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:35:42.691654 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:35:42.691654 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 30 00:35:42.691654 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:42.691654 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 234/464] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:42.691654 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce2313-0286-49e4-9672-3adc3072aad7 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:42.759283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f97b81-2425-41d5-86bf-23baf695d491 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:42.797748 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d08a9e-c518-4651-901c-87b8fe794cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:42.798357 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d08a9e-c518-4651-901c-87b8fe794cc6 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:42.798635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d08a9e-c518-4651-901c-87b8fe794cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:42.798900 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d08a9e-c518-4651-901c-87b8fe794cc6 None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:42.799628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d08a9e-c518-4651-901c-87b8fe794cc6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:42.799928 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d08a9e-c518-4651-901c-87b8fe794cc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:42.801390 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d08a9e-c518-4651-901c-87b8fe794cc6 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:42.813970 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce2313-0286-49e4-9672-3adc3072aad7 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:42.825763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 231/465] 10.222.0.36 () {56 vars in 1133 bytes} [Sun Oct 30 00:35:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:42.825763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d08a9e-c518-4651-901c-87b8fe794cc6 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:42.832724 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:42.849983 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:42.854688 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaed1e-df83-4956-a1ae-e1575e9fb95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:42.855125 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaed1e-df83-4956-a1ae-e1575e9fb95b None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:42.855406 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaed1e-df83-4956-a1ae-e1575e9fb95b None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:42.855894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aaed1e-df83-4956-a1ae-e1575e9fb95b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:42.863614 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aaed1e-df83-4956-a1ae-e1575e9fb95b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:42.865209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aaed1e-df83-4956-a1ae-e1575e9fb95b None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:42.879790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-e3aaed1e-df83-4956-a1ae-e1575e9fb95b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 30 00:35:42.879790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:42.879790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:42.879790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:42.879790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:42.879790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:42.879790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:42.879790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:42.879790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:42.879790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:42.879790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:42.879790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:42.879790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:35:42.879790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:35:42.879790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:35:42.879790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:35:42.880856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:42.880856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:42.880856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:35:42.880856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:42.880856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:35:42.880856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:42.880856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:42.880856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:42.880856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:42.880856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:42.880856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:42.880856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:42.880856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:42.880856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:42.880856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:42.880856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:42.880856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:42.880856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:42.881778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:42.881778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:42.881778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:35:42.881778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:35:42.881778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:35:42.881778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:35:42.881778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:35:42.881778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:35:42.881778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 30 00:35:42.881778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:42.881778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 235/466] 10.222.0.36 () {56 vars in 1102 bytes} [Sun Oct 30 00:35:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:42.881778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaed1e-df83-4956-a1ae-e1575e9fb95b None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:42.949930 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d08a9e-c518-4651-901c-87b8fe794cc6 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:42.986580 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16532e0-8c2d-4f75-9f9e-bf21023c94e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:42.987211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16532e0-8c2d-4f75-9f9e-bf21023c94e1 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:42.987494 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16532e0-8c2d-4f75-9f9e-bf21023c94e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:42.987763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16532e0-8c2d-4f75-9f9e-bf21023c94e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:42.988551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16532e0-8c2d-4f75-9f9e-bf21023c94e1 None admin] RBAC: Adding query filter params (role.id=9a8de43206fa4a8fa4901488bf86f9f5, scope.project.id=ab3a0e5fc5a344739b58ec8313639496, user.id=7cf9451f2ef7478589b4a4278cb03252) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:42.988863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16532e0-8c2d-4f75-9f9e-bf21023c94e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:42.990350 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16532e0-8c2d-4f75-9f9e-bf21023c94e1 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:42.997641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 232/467] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:35:42 2022] GET /identity/v3/role_assignments?user.id=7cf9451f2ef7478589b4a4278cb03252&scope.project.id=ab3a0e5fc5a344739b58ec8313639496&role.id=9a8de43206fa4a8fa4901488bf86f9f5 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:42.998125 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aaed1e-df83-4956-a1ae-e1575e9fb95b None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:42.998495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16532e0-8c2d-4f75-9f9e-bf21023c94e1 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:43.056204 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e70b1-c6ab-4078-be0b-f9bedeed3f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:43.057690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e70b1-c6ab-4078-be0b-f9bedeed3f1c None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:43.058208 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e70b1-c6ab-4078-be0b-f9bedeed3f1c None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:43.059413 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e70b1-c6ab-4078-be0b-f9bedeed3f1c None admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:43.060396 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e70b1-c6ab-4078-be0b-f9bedeed3f1c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:43.060854 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e70b1-c6ab-4078-be0b-f9bedeed3f1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:43.063581 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e70b1-c6ab-4078-be0b-f9bedeed3f1c None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:43.113991 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:43.122278 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 236/468] 10.222.0.36 () {56 vars in 1137 bytes} [Sun Oct 30 00:35:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:43.123709 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e70b1-c6ab-4078-be0b-f9bedeed3f1c None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:43.135973 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16532e0-8c2d-4f75-9f9e-bf21023c94e1 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:43.173404 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:43.179373 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1bf17-8575-4d67-b777-d0e2e967934a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:43.180261 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1bf17-8575-4d67-b777-d0e2e967934a None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:43.180717 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1bf17-8575-4d67-b777-d0e2e967934a None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:43.181129 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1bf17-8575-4d67-b777-d0e2e967934a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:43.265528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e70b1-c6ab-4078-be0b-f9bedeed3f1c None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:43.274202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1bf17-8575-4d67-b777-d0e2e967934a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:43.276309 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1bf17-8575-4d67-b777-d0e2e967934a None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:43.314733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d93553-2f8e-47d3-90d1-c53dfb4c123a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:43.315606 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d93553-2f8e-47d3-90d1-c53dfb4c123a None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:43.315903 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d93553-2f8e-47d3-90d1-c53dfb4c123a None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:43.316660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d93553-2f8e-47d3-90d1-c53dfb4c123a None admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:43.317687 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d93553-2f8e-47d3-90d1-c53dfb4c123a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:43.317996 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d93553-2f8e-47d3-90d1-c53dfb4c123a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:43.319719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d93553-2f8e-47d3-90d1-c53dfb4c123a None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:43.346761 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 237/469] 10.222.0.36 () {56 vars in 1139 bytes} [Sun Oct 30 00:35:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:43.347638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d93553-2f8e-47d3-90d1-c53dfb4c123a None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:43.378346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:43.397354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 233/470] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:35:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:35:43.398305 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1bf17-8575-4d67-b777-d0e2e967934a None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:43.462885 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:43.474279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d93553-2f8e-47d3-90d1-c53dfb4c123a None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:43.503767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1bf17-8575-4d67-b777-d0e2e967934a None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:43.513985 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebdd8b1-a551-4d4b-a18c-6ff4c02d5295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:43.514721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebdd8b1-a551-4d4b-a18c-6ff4c02d5295 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:43.515070 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebdd8b1-a551-4d4b-a18c-6ff4c02d5295 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:43.515534 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebdd8b1-a551-4d4b-a18c-6ff4c02d5295 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:43.547999 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b732e-18e6-41c1-8983-ccb0073a7ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:43.549338 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b732e-18e6-41c1-8983-ccb0073a7ee7 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:43.549794 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b732e-18e6-41c1-8983-ccb0073a7ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:43.550224 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b732e-18e6-41c1-8983-ccb0073a7ee7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:43.606200 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebdd8b1-a551-4d4b-a18c-6ff4c02d5295 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:43.608201 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebdd8b1-a551-4d4b-a18c-6ff4c02d5295 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:43.627418 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:43.646850 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b732e-18e6-41c1-8983-ccb0073a7ee7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:43.648700 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b732e-18e6-41c1-8983-ccb0073a7ee7 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:43.738900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:43.760072 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 238/471] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:35:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:35:43.760072 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebdd8b1-a551-4d4b-a18c-6ff4c02d5295 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:43.809778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 234/472] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:35:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 30 00:35:43.810001 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b732e-18e6-41c1-8983-ccb0073a7ee7 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:43.908772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebdd8b1-a551-4d4b-a18c-6ff4c02d5295 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:43.925224 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:43.944070 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b732e-18e6-41c1-8983-ccb0073a7ee7 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:43.958676 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac226b-a46e-4432-b745-5f58a3088b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:43.959769 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac226b-a46e-4432-b745-5f58a3088b4f None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:43.960289 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac226b-a46e-4432-b745-5f58a3088b4f None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:43.960803 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac226b-a46e-4432-b745-5f58a3088b4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:43.973027 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac226b-a46e-4432-b745-5f58a3088b4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:43.975106 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac226b-a46e-4432-b745-5f58a3088b4f None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:43.982588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df288ad-53c0-4058-aceb-7eaf3c80a751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:43.983321 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df288ad-53c0-4058-aceb-7eaf3c80a751 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:43.983602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df288ad-53c0-4058-aceb-7eaf3c80a751 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:43.983934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df288ad-53c0-4058-aceb-7eaf3c80a751 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:43.994435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-04ac226b-a46e-4432-b745-5f58a3088b4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:43.994435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:43.994435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:43.994435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:43.994435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:43.994435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:43.994435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:43.994435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:43.994435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:43.994435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:43.994435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:43.994435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:43.994435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:43.994435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:43.994435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:43.994435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:43.996076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:43.996076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:43.996076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:43.996076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:43.996076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:43.996076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:43.996076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:43.996076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:43.996076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:43.996076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:43.996076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:43.996076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:43.996076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:43.996076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:43.996076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:43.996076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:43.996076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:43.997246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:43.997246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:43.997246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:43.997419 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df288ad-53c0-4058-aceb-7eaf3c80a751 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:43.997419 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df288ad-53c0-4058-aceb-7eaf3c80a751 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:43.999231 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 239/473] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:44.000371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac226b-a46e-4432-b745-5f58a3088b4f None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:44.013929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-5df288ad-53c0-4058-aceb-7eaf3c80a751 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:44.013929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:44.013929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:44.013929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:44.013929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:44.013929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:44.013929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:44.013929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:44.013929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:44.013929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:44.013929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:44.013929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:44.013929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:44.013929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:44.013929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:44.013929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:44.014879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:44.014879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:44.014879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:44.014879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:44.014879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:44.014879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:44.014879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:44.014879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:44.014879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:44.014879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:44.014879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:44.014879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:44.014879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:44.014879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:44.014879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:44.014879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:44.014879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:44.015721 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:44.015721 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:44.015721 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:44.020012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 235/474] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:44.020848 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df288ad-53c0-4058-aceb-7eaf3c80a751 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:44.058441 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:44.124453 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac226b-a46e-4432-b745-5f58a3088b4f None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:44.136138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df288ad-53c0-4058-aceb-7eaf3c80a751 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:44.167395 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2814d-e0f7-4355-a581-388145962eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:44.168624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2814d-e0f7-4355-a581-388145962eb0 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:44.169206 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2814d-e0f7-4355-a581-388145962eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:44.169685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2814d-e0f7-4355-a581-388145962eb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:44.174412 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988033ac-a762-432d-95e4-15b32bab2f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:44.175076 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988033ac-a762-432d-95e4-15b32bab2f70 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:44.175362 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988033ac-a762-432d-95e4-15b32bab2f70 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:44.175635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988033ac-a762-432d-95e4-15b32bab2f70 None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:44.176314 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988033ac-a762-432d-95e4-15b32bab2f70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:44.176619 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988033ac-a762-432d-95e4-15b32bab2f70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:44.178061 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988033ac-a762-432d-95e4-15b32bab2f70 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:44.180762 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:44.186115 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2814d-e0f7-4355-a581-388145962eb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:44.188801 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2814d-e0f7-4355-a581-388145962eb0 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:44.193967 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 236/475] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:44.194940 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988033ac-a762-432d-95e4-15b32bab2f70 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:44.207817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-9eb2814d-e0f7-4355-a581-388145962eb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:44.207817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:44.207817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:44.207817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:44.207817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:44.207817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:44.207817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:44.207817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:44.207817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:44.207817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:44.207817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:44.207817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:44.207817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:44.207817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:44.207817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:44.207817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:44.208839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:44.208839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:44.208839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:44.208839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:44.208839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:44.208839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:44.208839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:44.208839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:44.208839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:44.208839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:44.208839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:44.208839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:44.208839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:44.208839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:44.208839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:44.208839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:44.208839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:44.209692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:44.209692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:44.209692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:44.212017 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 240/476] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:44.214084 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2814d-e0f7-4355-a581-388145962eb0 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:44.314133 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988033ac-a762-432d-95e4-15b32bab2f70 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:44.328947 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:44.337815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2814d-e0f7-4355-a581-388145962eb0 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:44.352437 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28fa3f-c2fd-474f-9e70-373fcd8f3b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:44.353148 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28fa3f-c2fd-474f-9e70-373fcd8f3b3b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:44.354016 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28fa3f-c2fd-474f-9e70-373fcd8f3b3b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:44.354454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28fa3f-c2fd-474f-9e70-373fcd8f3b3b None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:44.355109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28fa3f-c2fd-474f-9e70-373fcd8f3b3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:44.355413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28fa3f-c2fd-474f-9e70-373fcd8f3b3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:44.356836 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28fa3f-c2fd-474f-9e70-373fcd8f3b3b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:44.371476 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 237/477] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:44.372162 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28fa3f-c2fd-474f-9e70-373fcd8f3b3b None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:44.384593 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466b145-82f2-4848-8f5d-644b7b01e660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:44.386915 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466b145-82f2-4848-8f5d-644b7b01e660 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:44.387770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466b145-82f2-4848-8f5d-644b7b01e660 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:44.388247 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466b145-82f2-4848-8f5d-644b7b01e660 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:44.397252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d466b145-82f2-4848-8f5d-644b7b01e660 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:44.398728 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d466b145-82f2-4848-8f5d-644b7b01e660 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:44.411522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-d466b145-82f2-4848-8f5d-644b7b01e660 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:44.411522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:44.411522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:44.411522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:44.411522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:44.411522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:44.411522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:44.411522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:44.411522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:44.411522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:44.411522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:44.411522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:44.411522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:35:44.411522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:35:44.411522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:35:44.412494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:35:44.412494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:44.412494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:44.412494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:44.412494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:44.412494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:44.412494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:44.412494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:44.412494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:44.412494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:44.412494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:44.412494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:44.412494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:44.412494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:44.412494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:44.412494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:44.412494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:44.413494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:35:44.413494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:35:44.413494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:35:44.413494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:35:44.413494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:35:44.413494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:35:44.413494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:44.413494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:44.413494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 241/478] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:44.413958 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcf7f9-0404-46af-9fad-14dbef242687 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:44.415126 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcf7f9-0404-46af-9fad-14dbef242687 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:44.415126 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcf7f9-0404-46af-9fad-14dbef242687 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:44.416649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 242/479] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:44.417613 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcf7f9-0404-46af-9fad-14dbef242687 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:44.451216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:44.499848 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28fa3f-c2fd-474f-9e70-373fcd8f3b3b None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:44.526413 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcf7f9-0404-46af-9fad-14dbef242687 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:44.536903 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f561e20-3879-4cdb-9ccd-fd24b485dd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:44.537842 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f561e20-3879-4cdb-9ccd-fd24b485dd0c None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:44.538350 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f561e20-3879-4cdb-9ccd-fd24b485dd0c None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:44.538552 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f561e20-3879-4cdb-9ccd-fd24b485dd0c None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:44.539194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f561e20-3879-4cdb-9ccd-fd24b485dd0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:44.539749 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f561e20-3879-4cdb-9ccd-fd24b485dd0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:44.541770 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f561e20-3879-4cdb-9ccd-fd24b485dd0c None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:44.554586 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 238/480] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:44.555305 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f561e20-3879-4cdb-9ccd-fd24b485dd0c None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:44.569666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39b0a5-3f56-4c2c-bca1-030de9889f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:44.570305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39b0a5-3f56-4c2c-bca1-030de9889f97 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:44.570590 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39b0a5-3f56-4c2c-bca1-030de9889f97 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:44.570867 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39b0a5-3f56-4c2c-bca1-030de9889f97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:44.579017 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39b0a5-3f56-4c2c-bca1-030de9889f97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:44.580579 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39b0a5-3f56-4c2c-bca1-030de9889f97 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:44.597588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-bc39b0a5-3f56-4c2c-bca1-030de9889f97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:44.597588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:44.597588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:44.597588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:44.597588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:44.597588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:44.597588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:44.597588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:44.597588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:44.597588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:44.597588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:44.597588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:44.597588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:35:44.597588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:35:44.597588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:35:44.598487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:35:44.598487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:44.598487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:44.598487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:44.598487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:44.598487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:44.598487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:44.598487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:44.598487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:44.598487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:44.598487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:44.598487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:44.598487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:44.598487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:44.598487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:44.598487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:44.598487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:44.599412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:35:44.599412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:35:44.599412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:35:44.599412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:35:44.599412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:35:44.599412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:35:44.599412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:44.599412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:44.599412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 243/481] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:44.599871 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f353d05-ac68-4c76-afb1-9feaf1e0f123 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:44.600498 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f353d05-ac68-4c76-afb1-9feaf1e0f123 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:44.600498 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f353d05-ac68-4c76-afb1-9feaf1e0f123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:44.619520 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-3f353d05-ac68-4c76-afb1-9feaf1e0f123 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:44.620785 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-3f353d05-ac68-4c76-afb1-9feaf1e0f123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:44.636343 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:44.678058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f561e20-3879-4cdb-9ccd-fd24b485dd0c None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:44.696250 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:44.709482 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 244/482] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:44.710571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f353d05-ac68-4c76-afb1-9feaf1e0f123 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:44.722957 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6cf3e-ed6f-4b39-9c27-50698a0aa543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:44.723893 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6cf3e-ed6f-4b39-9c27-50698a0aa543 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:44.724401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6cf3e-ed6f-4b39-9c27-50698a0aa543 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:44.724868 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6cf3e-ed6f-4b39-9c27-50698a0aa543 None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:44.725778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6cf3e-ed6f-4b39-9c27-50698a0aa543 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:44.726275 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6cf3e-ed6f-4b39-9c27-50698a0aa543 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:44.727908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6cf3e-ed6f-4b39-9c27-50698a0aa543 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:44.745365 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 239/483] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:35:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:44.746571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6cf3e-ed6f-4b39-9c27-50698a0aa543 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:44.842722 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f353d05-ac68-4c76-afb1-9feaf1e0f123 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:44.865063 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6cf3e-ed6f-4b39-9c27-50698a0aa543 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:44.894932 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9efaf-e5d0-4804-a8a1-3e64d56015a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:44.896264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9efaf-e5d0-4804-a8a1-3e64d56015a1 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:44.896774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9efaf-e5d0-4804-a8a1-3e64d56015a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:44.897191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9efaf-e5d0-4804-a8a1-3e64d56015a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:44.908597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9efaf-e5d0-4804-a8a1-3e64d56015a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:44.911453 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9efaf-e5d0-4804-a8a1-3e64d56015a1 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:44.915722 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32c79a-b3fa-4d87-a323-020fb9a2213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:44.916787 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32c79a-b3fa-4d87-a323-020fb9a2213b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:44.917166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32c79a-b3fa-4d87-a323-020fb9a2213b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:44.917540 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32c79a-b3fa-4d87-a323-020fb9a2213b None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:44.918348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c32c79a-b3fa-4d87-a323-020fb9a2213b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:44.918755 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c32c79a-b3fa-4d87-a323-020fb9a2213b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:44.920375 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c32c79a-b3fa-4d87-a323-020fb9a2213b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:44.927246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-99c9efaf-e5d0-4804-a8a1-3e64d56015a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:44.927246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:44.927246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:44.927246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:44.927246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:44.927246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:44.927246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:44.927246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:44.927246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:44.927246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:44.927246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:44.927246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:44.927246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:35:44.927246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:35:44.927246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:35:44.928405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:35:44.928405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:44.928405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:44.928405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:44.928405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:44.928405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:44.928405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:44.928405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:44.928405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:44.928405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:44.928405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:44.928405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:44.928405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:44.928405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:44.928405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:44.928405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:44.928405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:44.929468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:35:44.929468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:35:44.929468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:35:44.929468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:35:44.929468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:35:44.929468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:35:44.929468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:44.929468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:44.933003 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 245/484] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:44.935603 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9efaf-e5d0-4804-a8a1-3e64d56015a1 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:44.939132 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 240/485] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:35:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:44.940861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32c79a-b3fa-4d87-a323-020fb9a2213b None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:44.947772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:44.987584 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:45.071196 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32c79a-b3fa-4d87-a323-020fb9a2213b None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:45.080281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9efaf-e5d0-4804-a8a1-3e64d56015a1 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:45.109286 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa557796-f29e-499c-948a-b42fef196bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:45.109912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa557796-f29e-499c-948a-b42fef196bde None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:45.110175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa557796-f29e-499c-948a-b42fef196bde None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:45.110445 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa557796-f29e-499c-948a-b42fef196bde None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:45.111144 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa557796-f29e-499c-948a-b42fef196bde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:45.111444 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa557796-f29e-499c-948a-b42fef196bde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:45.113114 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa557796-f29e-499c-948a-b42fef196bde None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:45.121964 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ac7b8-ae49-419c-8734-927c0daae627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:45.122678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ac7b8-ae49-419c-8734-927c0daae627 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:45.123042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ac7b8-ae49-419c-8734-927c0daae627 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:45.123382 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ac7b8-ae49-419c-8734-927c0daae627 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:45.124256 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ac7b8-ae49-419c-8734-927c0daae627 None admin] RBAC: Adding query filter params (role.id=9a8de43206fa4a8fa4901488bf86f9f5, scope.project.id=ab3a0e5fc5a344739b58ec8313639496, user.id=adbba7be39a64230bab160ce77ce7586) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:45.124641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ac7b8-ae49-419c-8734-927c0daae627 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:45.126108 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ac7b8-ae49-419c-8734-927c0daae627 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:45.132046 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 241/486] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:35:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:45.133176 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa557796-f29e-499c-948a-b42fef196bde None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:45.138731 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 246/487] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:35:44 2022] GET /identity/v3/role_assignments?user.id=adbba7be39a64230bab160ce77ce7586&scope.project.id=ab3a0e5fc5a344739b58ec8313639496&role.id=9a8de43206fa4a8fa4901488bf86f9f5 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:45.145311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ac7b8-ae49-419c-8734-927c0daae627 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:45.230381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:45.284824 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:45.294357 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa557796-f29e-499c-948a-b42fef196bde None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:45.309801 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ac7b8-ae49-419c-8734-927c0daae627 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:45.351482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef1fee-0ede-43b8-9998-5c3edf07c4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:45.351860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef1fee-0ede-43b8-9998-5c3edf07c4c4 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:45.352191 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef1fee-0ede-43b8-9998-5c3edf07c4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:45.352467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef1fee-0ede-43b8-9998-5c3edf07c4c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:45.353210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef1fee-0ede-43b8-9998-5c3edf07c4c4 None admin] RBAC: Adding query filter params (role.id=9a8de43206fa4a8fa4901488bf86f9f5, scope.project.id=ab3a0e5fc5a344739b58ec8313639496, user.id=2dbe6b3bdfb3405589b8ce5aebebb93e) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:45.354158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef1fee-0ede-43b8-9998-5c3edf07c4c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:45.355624 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef1fee-0ede-43b8-9998-5c3edf07c4c4 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:45.362737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 242/488] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:35:45 2022] GET /identity/v3/role_assignments?user.id=2dbe6b3bdfb3405589b8ce5aebebb93e&scope.project.id=ab3a0e5fc5a344739b58ec8313639496&role.id=9a8de43206fa4a8fa4901488bf86f9f5 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:45.363617 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef1fee-0ede-43b8-9998-5c3edf07c4c4 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:45.366207 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77172baf-89cf-4ba6-b32d-8141e246754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:45.367294 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77172baf-89cf-4ba6-b32d-8141e246754d None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:45.368748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77172baf-89cf-4ba6-b32d-8141e246754d None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:45.369238 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77172baf-89cf-4ba6-b32d-8141e246754d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:45.385103 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77172baf-89cf-4ba6-b32d-8141e246754d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:45.390796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-77172baf-89cf-4ba6-b32d-8141e246754d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:45.390796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:45.390796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:45.390796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:45.390796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:45.390796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:45.390796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:45.390796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:45.390796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:45.390796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:45.390796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:45.390796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:45.390796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:45.390796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:45.390796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:45.390796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:45.391954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:45.391954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:45.391954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:45.391954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:45.391954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:45.391954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:45.391954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:45.391954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:45.391954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:45.391954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:45.391954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:45.391954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:45.391954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:45.391954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:45.391954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:45.391954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:45.392944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:45.392944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:45.392944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:45.392944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:45.392944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:45.392944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:45.392944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:45.392944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:45.392944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:45.392944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:45.392944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:45.392944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:45.392944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 247/489] 10.222.0.36 () {56 vars in 1102 bytes} [Sun Oct 30 00:35:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:45.398724 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77172baf-89cf-4ba6-b32d-8141e246754d None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:45.509685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef1fee-0ede-43b8-9998-5c3edf07c4c4 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:45.547685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77172baf-89cf-4ba6-b32d-8141e246754d None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:45.558774 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f490e9-dbfa-4e52-b1a8-801ba9c89d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:45.559544 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f490e9-dbfa-4e52-b1a8-801ba9c89d44 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:45.559816 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f490e9-dbfa-4e52-b1a8-801ba9c89d44 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:45.560198 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f490e9-dbfa-4e52-b1a8-801ba9c89d44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:45.561064 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f490e9-dbfa-4e52-b1a8-801ba9c89d44 None admin] RBAC: Adding query filter params (role.id=9a8de43206fa4a8fa4901488bf86f9f5, scope.project.id=ab3a0e5fc5a344739b58ec8313639496, user.id=fb39b6cc97a1456bb1312f49118c6e10) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:45.561371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f490e9-dbfa-4e52-b1a8-801ba9c89d44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:45.562874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f490e9-dbfa-4e52-b1a8-801ba9c89d44 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:45.571076 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 243/490] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:35:45 2022] GET /identity/v3/role_assignments?user.id=fb39b6cc97a1456bb1312f49118c6e10&scope.project.id=ab3a0e5fc5a344739b58ec8313639496&role.id=9a8de43206fa4a8fa4901488bf86f9f5 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:45.573957 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a5e3f-81b9-415d-85bd-59d2f0fa299d None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:45.574236 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a5e3f-81b9-415d-85bd-59d2f0fa299d None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:45.574516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a5e3f-81b9-415d-85bd-59d2f0fa299d None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:45.577313 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 244/491] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:45.585602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584d2a3-81cb-4006-98e9-5ddff1716f2d None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:45.585888 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584d2a3-81cb-4006-98e9-5ddff1716f2d None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:45.586175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584d2a3-81cb-4006-98e9-5ddff1716f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:45.602928 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:45.611774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82ade6-a970-47f7-9f9b-2b768d5cd5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:45.612493 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82ade6-a970-47f7-9f9b-2b768d5cd5b4 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:45.612770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82ade6-a970-47f7-9f9b-2b768d5cd5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:45.613047 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82ade6-a970-47f7-9f9b-2b768d5cd5b4 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:45.613729 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-c584d2a3-81cb-4006-98e9-5ddff1716f2d None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:45.613876 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82ade6-a970-47f7-9f9b-2b768d5cd5b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:45.614039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82ade6-a970-47f7-9f9b-2b768d5cd5b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:45.615024 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c584d2a3-81cb-4006-98e9-5ddff1716f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:45.615488 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82ade6-a970-47f7-9f9b-2b768d5cd5b4 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:45.626956 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 248/492] 10.222.0.36 () {56 vars in 1103 bytes} [Sun Oct 30 00:35:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:45.634524 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82ade6-a970-47f7-9f9b-2b768d5cd5b4 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:45.643437 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:45.708566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 245/493] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:45.760649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82ade6-a970-47f7-9f9b-2b768d5cd5b4 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:45.800488 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97a751-123b-4d44-908c-644a0703fdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:45.801296 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97a751-123b-4d44-908c-644a0703fdc6 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:45.801676 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97a751-123b-4d44-908c-644a0703fdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:45.802077 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97a751-123b-4d44-908c-644a0703fdc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:45.816619 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97a751-123b-4d44-908c-644a0703fdc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:45.818250 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97a751-123b-4d44-908c-644a0703fdc6 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:45.834369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-1f97a751-123b-4d44-908c-644a0703fdc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:45.834369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:45.834369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:45.834369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:45.834369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:45.834369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:45.834369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:45.834369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:45.834369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:45.834369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:45.834369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:45.834369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:45.834369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:45.834369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:45.834369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:45.834369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:45.837320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:45.837320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:45.837320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:45.837320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:45.837320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:45.837320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:45.837320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:45.837320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:45.837320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:45.837320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:45.837320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:45.837320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:45.837320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:45.837320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:45.837320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:45.837320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:45.837320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:45.839878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:45.839878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:45.839878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:45.839878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 249/494] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:45.848159 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584d2a3-81cb-4006-98e9-5ddff1716f2d None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:45.951356 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584d2a3-81cb-4006-98e9-5ddff1716f2d None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:45.989909 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11441b51-b572-4075-bfff-eb46fd214619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:45.990690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11441b51-b572-4075-bfff-eb46fd214619 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:45.991065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11441b51-b572-4075-bfff-eb46fd214619 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:45.991443 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11441b51-b572-4075-bfff-eb46fd214619 None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:45.992223 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11441b51-b572-4075-bfff-eb46fd214619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:45.992627 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11441b51-b572-4075-bfff-eb46fd214619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:45.994744 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11441b51-b572-4075-bfff-eb46fd214619 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:46.008062 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 246/495] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:46.014660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97a751-123b-4d44-908c-644a0703fdc6 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:46.030201 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11441b51-b572-4075-bfff-eb46fd214619 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:46.061150 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:46.104392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:46.224939 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97a751-123b-4d44-908c-644a0703fdc6 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:46.239493 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11441b51-b572-4075-bfff-eb46fd214619 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:46.270497 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60078cc8-551c-41e5-8b40-5d99a4441be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:46.271254 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60078cc8-551c-41e5-8b40-5d99a4441be9 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:46.271724 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60078cc8-551c-41e5-8b40-5d99a4441be9 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:46.272219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60078cc8-551c-41e5-8b40-5d99a4441be9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:46.288958 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60078cc8-551c-41e5-8b40-5d99a4441be9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:46.290831 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60078cc8-551c-41e5-8b40-5d99a4441be9 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:46.316882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-60078cc8-551c-41e5-8b40-5d99a4441be9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 30 00:35:46.316882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:46.316882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:46.316882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:46.316882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:46.316882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:46.316882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:46.316882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:46.316882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:46.316882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:46.316882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:46.316882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:46.316882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:35:46.316882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:35:46.316882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:35:46.316882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:35:46.318117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:46.318117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:46.318117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:35:46.318117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:46.318117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:35:46.318117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:46.318117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:46.318117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:46.318117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:46.318117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:46.318117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:46.318117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:46.318117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:46.318117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:46.318117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:46.318117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:46.318117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:46.318117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:46.319326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:46.319326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:46.319326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:35:46.319326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:35:46.319326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:35:46.319326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:35:46.319326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:35:46.319326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:35:46.319326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 30 00:35:46.319326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:46.321490 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 250/496] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:35:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:46.324251 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac62bf1-cf3d-4339-a2c6-2e8627774947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:46.324993 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac62bf1-cf3d-4339-a2c6-2e8627774947 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:46.325428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac62bf1-cf3d-4339-a2c6-2e8627774947 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:46.325811 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac62bf1-cf3d-4339-a2c6-2e8627774947 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:46.327449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60078cc8-551c-41e5-8b40-5d99a4441be9 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:46.350748 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac62bf1-cf3d-4339-a2c6-2e8627774947 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:46.352500 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac62bf1-cf3d-4339-a2c6-2e8627774947 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:46.370884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-dac62bf1-cf3d-4339-a2c6-2e8627774947 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:46.370884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:46.370884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:46.370884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:46.370884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:46.370884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:46.370884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:46.370884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:46.370884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:46.370884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:46.370884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:46.370884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:46.370884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:46.370884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:46.370884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:46.370884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:46.372202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:46.372202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:46.372202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:46.372202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:46.372202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:46.372202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:46.372202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:46.372202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:46.372202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:46.372202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:46.372202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:46.372202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:46.372202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:46.372202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:46.372202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:46.372202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:46.372202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:46.373369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:46.373369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:46.373369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:46.373588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 247/497] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:35:46.386623 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac62bf1-cf3d-4339-a2c6-2e8627774947 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:46.500247 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:46.543855 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60078cc8-551c-41e5-8b40-5d99a4441be9 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:46.547434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:46.571748 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac62bf1-cf3d-4339-a2c6-2e8627774947 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:46.613939 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b8784-04ca-4e08-92f6-19ea2ecf4025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:46.614558 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b8784-04ca-4e08-92f6-19ea2ecf4025 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:46.614839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b8784-04ca-4e08-92f6-19ea2ecf4025 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:46.615109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b8784-04ca-4e08-92f6-19ea2ecf4025 None admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:46.615921 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b8784-04ca-4e08-92f6-19ea2ecf4025 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:46.616255 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b8784-04ca-4e08-92f6-19ea2ecf4025 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:46.617724 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b8784-04ca-4e08-92f6-19ea2ecf4025 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:46.643990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b302d8-9711-4a25-b338-184886930f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:46.644644 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b302d8-9711-4a25-b338-184886930f19 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:46.644927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b302d8-9711-4a25-b338-184886930f19 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:46.645203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b302d8-9711-4a25-b338-184886930f19 None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:46.645868 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b302d8-9711-4a25-b338-184886930f19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:46.646173 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b302d8-9711-4a25-b338-184886930f19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:46.647562 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b302d8-9711-4a25-b338-184886930f19 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:46.651386 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 251/498] 10.222.0.36 () {56 vars in 1135 bytes} [Sun Oct 30 00:35:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:46.658000 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b8784-04ca-4e08-92f6-19ea2ecf4025 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:46.676424 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 248/499] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:46.682812 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b302d8-9711-4a25-b338-184886930f19 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:46.800863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b8784-04ca-4e08-92f6-19ea2ecf4025 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:46.838946 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765ff87-c487-44ca-8b62-2d0af85d4244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:46.839574 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765ff87-c487-44ca-8b62-2d0af85d4244 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:46.839857 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765ff87-c487-44ca-8b62-2d0af85d4244 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:46.840260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765ff87-c487-44ca-8b62-2d0af85d4244 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:46.873850 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:46.907885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b302d8-9711-4a25-b338-184886930f19 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:46.918093 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765ff87-c487-44ca-8b62-2d0af85d4244 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:46.919605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765ff87-c487-44ca-8b62-2d0af85d4244 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:46.971143 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:46.989232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8be1a6-5bf9-4f9b-b766-94e273524b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:46.989902 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8be1a6-5bf9-4f9b-b766-94e273524b3a None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:46.990226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8be1a6-5bf9-4f9b-b766-94e273524b3a None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:46.990517 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8be1a6-5bf9-4f9b-b766-94e273524b3a None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:46.991175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8be1a6-5bf9-4f9b-b766-94e273524b3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:46.992860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8be1a6-5bf9-4f9b-b766-94e273524b3a None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:47.024622 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 249/500] 10.222.0.36 () {60 vars in 1132 bytes} [Sun Oct 30 00:35:46 2022] POST /identity/v3/users => generated 254 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:35:47.091745 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 252/501] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:35:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 30 00:35:47.097048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8be1a6-5bf9-4f9b-b766-94e273524b3a None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:47.243790 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8be1a6-5bf9-4f9b-b766-94e273524b3a None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:47.287488 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5ddc3-3719-4761-a1ec-cc2626d41880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:47.288225 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5ddc3-3719-4761-a1ec-cc2626d41880 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:47.288509 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5ddc3-3719-4761-a1ec-cc2626d41880 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:47.288778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5ddc3-3719-4761-a1ec-cc2626d41880 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:47.301206 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5ddc3-3719-4761-a1ec-cc2626d41880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:47.302752 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5ddc3-3719-4761-a1ec-cc2626d41880 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:47.318335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-30d5ddc3-3719-4761-a1ec-cc2626d41880 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:47.318335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:47.318335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:47.318335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:47.318335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:47.318335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:47.318335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:47.318335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:47.318335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:47.318335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:47.318335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:47.318335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:47.318335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:47.318335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:47.318335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:47.318335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:47.320068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:47.320068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:47.320068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:47.320068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:47.320068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:47.320068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:47.320068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:47.320068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:47.320068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:47.320068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:47.320068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:47.320068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:47.320068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:47.320068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:47.320068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:47.320068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:47.320068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:47.321222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:47.321222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:47.321222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:47.321222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 250/502] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:47.328096 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765ff87-c487-44ca-8b62-2d0af85d4244 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:47.396726 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:47.448786 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765ff87-c487-44ca-8b62-2d0af85d4244 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:47.496898 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a0af4-2c3b-48df-984d-260296187d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:47.497527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a0af4-2c3b-48df-984d-260296187d6f None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:47.497809 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a0af4-2c3b-48df-984d-260296187d6f None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:47.498086 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a0af4-2c3b-48df-984d-260296187d6f None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:47.498743 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a0af4-2c3b-48df-984d-260296187d6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:47.499054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a0af4-2c3b-48df-984d-260296187d6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:47.500477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a0af4-2c3b-48df-984d-260296187d6f None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:47.516409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 253/503] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:47.526081 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5ddc3-3719-4761-a1ec-cc2626d41880 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:47.589936 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:47.678540 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5ddc3-3719-4761-a1ec-cc2626d41880 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:47.725861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78263bb8-9788-47b0-b4a3-087d26721810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:47.726557 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78263bb8-9788-47b0-b4a3-087d26721810 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:47.726770 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78263bb8-9788-47b0-b4a3-087d26721810 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:47.727044 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78263bb8-9788-47b0-b4a3-087d26721810 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:47.735196 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78263bb8-9788-47b0-b4a3-087d26721810 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:47.736748 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78263bb8-9788-47b0-b4a3-087d26721810 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:47.758413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-78263bb8-9788-47b0-b4a3-087d26721810 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:47.758413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:47.758413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:47.758413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:47.758413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:47.758413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:47.758413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:47.758413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:47.758413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:47.758413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:47.758413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:47.758413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:47.758413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:35:47.758413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:35:47.758413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:35:47.759408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:35:47.759408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:47.759408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:47.759408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:47.759408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:47.759408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:47.759408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:47.759408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:47.759408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:47.759408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:47.759408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:47.759408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:47.759408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:47.759408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:47.759408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:47.759408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:47.759408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:47.761184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:35:47.761184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:35:47.761184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:35:47.761184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:35:47.761184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:35:47.761184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:35:47.761184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:47.761184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:47.761184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 251/504] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:35:47.768460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a0af4-2c3b-48df-984d-260296187d6f None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:47.979419 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a0af4-2c3b-48df-984d-260296187d6f None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:48.018221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:48.032392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec253b-d5f1-4e2e-b7d9-c4aee1c4f9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:48.033009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec253b-d5f1-4e2e-b7d9-c4aee1c4f9c8 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:48.033287 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec253b-d5f1-4e2e-b7d9-c4aee1c4f9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:48.033559 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec253b-d5f1-4e2e-b7d9-c4aee1c4f9c8 None admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:48.034257 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec253b-d5f1-4e2e-b7d9-c4aee1c4f9c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:48.034561 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec253b-d5f1-4e2e-b7d9-c4aee1c4f9c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:48.036011 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec253b-d5f1-4e2e-b7d9-c4aee1c4f9c8 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:48.054654 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 254/505] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:35:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:48.060742 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78263bb8-9788-47b0-b4a3-087d26721810 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:48.116122 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21c217-5091-41ba-a5d5-9762f6ebc243 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:48.116487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21c217-5091-41ba-a5d5-9762f6ebc243 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:48.116674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21c217-5091-41ba-a5d5-9762f6ebc243 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:48.118308 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 255/506] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:35:48.124403 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75cc19-d8a8-40c0-ad97-06f2a60b3e3e None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:48.124754 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75cc19-d8a8-40c0-ad97-06f2a60b3e3e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:48.124954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75cc19-d8a8-40c0-ad97-06f2a60b3e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:48.153907 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-8d75cc19-d8a8-40c0-ad97-06f2a60b3e3e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:48.155219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-8d75cc19-d8a8-40c0-ad97-06f2a60b3e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:48.196761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:48.203161 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78263bb8-9788-47b0-b4a3-087d26721810 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:48.234646 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 256/507] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:48.240448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f75519-6720-4682-b1a2-9e63ccff8c09 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:48.240933 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f75519-6720-4682-b1a2-9e63ccff8c09 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:48.241410 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f75519-6720-4682-b1a2-9e63ccff8c09 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:48.242627 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191315e-33e2-4478-a4c1-12b10dab23b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:48.243302 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 257/508] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:48.243432 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191315e-33e2-4478-a4c1-12b10dab23b2 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:48.247982 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191315e-33e2-4478-a4c1-12b10dab23b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:48.248398 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191315e-33e2-4478-a4c1-12b10dab23b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:48.249276 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191315e-33e2-4478-a4c1-12b10dab23b2 None admin] RBAC: Adding query filter params (role.id=9a8de43206fa4a8fa4901488bf86f9f5, scope.project.id=ab3a0e5fc5a344739b58ec8313639496, user.id=9f4c779b8422432fb06cdf996841b9b5) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:48.249689 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191315e-33e2-4478-a4c1-12b10dab23b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:48.250022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0c35e-d79b-42b5-b716-e7f7e4474410 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:48.250458 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0c35e-d79b-42b5-b716-e7f7e4474410 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:48.250939 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0c35e-d79b-42b5-b716-e7f7e4474410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:48.251511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191315e-33e2-4478-a4c1-12b10dab23b2 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:48.258513 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 252/509] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:35:48 2022] GET /identity/v3/role_assignments?user.id=9f4c779b8422432fb06cdf996841b9b5&scope.project.id=ab3a0e5fc5a344739b58ec8313639496&role.id=9a8de43206fa4a8fa4901488bf86f9f5 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:48.288155 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-24e0c35e-d79b-42b5-b716-e7f7e4474410 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:48.288155 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-24e0c35e-d79b-42b5-b716-e7f7e4474410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:48.441046 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 258/510] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:35:48.528650 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02fcc4-4bbf-43ad-99b6-2c0f1978b9d0 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:48.528834 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02fcc4-4bbf-43ad-99b6-2c0f1978b9d0 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:48.529113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02fcc4-4bbf-43ad-99b6-2c0f1978b9d0 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:48.530793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 253/511] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:48.536428 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546d048-3563-4c17-9db9-49822f5cf41b None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:48.536720 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546d048-3563-4c17-9db9-49822f5cf41b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:48.536984 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546d048-3563-4c17-9db9-49822f5cf41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:48.564397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-7546d048-3563-4c17-9db9-49822f5cf41b None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:48.565602 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-7546d048-3563-4c17-9db9-49822f5cf41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:48.594400 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02fcc4-4bbf-43ad-99b6-2c0f1978b9d0 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:48.625042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:48.662021 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 259/512] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:48.724542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02fcc4-4bbf-43ad-99b6-2c0f1978b9d0 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:48.741124 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546d048-3563-4c17-9db9-49822f5cf41b None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:48.769157 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee163fb4-6ed7-4426-a2b0-eaa80336a23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:48.770171 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee163fb4-6ed7-4426-a2b0-eaa80336a23e None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:48.770735 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee163fb4-6ed7-4426-a2b0-eaa80336a23e None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:48.771176 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee163fb4-6ed7-4426-a2b0-eaa80336a23e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:48.792132 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee163fb4-6ed7-4426-a2b0-eaa80336a23e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:48.804122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-ee163fb4-6ed7-4426-a2b0-eaa80336a23e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:35:48.804122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:48.804122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:48.804122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:48.804122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:48.804122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:48.804122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:48.804122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:48.804122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:48.804122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:48.804122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:48.804122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:48.804122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:48.804122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:48.804122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:48.804122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:48.805337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:48.805337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:48.805337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:48.805337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:48.805337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:48.805337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:48.805337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:48.805337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:48.805337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:48.805337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:48.805337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:48.805337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:48.805337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:48.805337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:48.805337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:48.805337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:48.806367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:48.806367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:48.806367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:48.806367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:48.806367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:48.806367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:48.806367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:48.806367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:48.806367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:48.806367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:48.806367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:35:48.806367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:48.809628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 254/513] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:35:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:35:48.815479 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee163fb4-6ed7-4426-a2b0-eaa80336a23e None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:48.853451 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:48.892627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546d048-3563-4c17-9db9-49822f5cf41b None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:48.931219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c66eaff-ad58-4edd-8b9a-bd8480fa35dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:48.932100 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66eaff-ad58-4edd-8b9a-bd8480fa35dd None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:48.932499 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66eaff-ad58-4edd-8b9a-bd8480fa35dd None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:48.932872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66eaff-ad58-4edd-8b9a-bd8480fa35dd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:48.933579 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c66eaff-ad58-4edd-8b9a-bd8480fa35dd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:48.935094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c66eaff-ad58-4edd-8b9a-bd8480fa35dd None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:48.946700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-5c66eaff-ad58-4edd-8b9a-bd8480fa35dd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 30 00:35:48.946700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:48.946700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:35:48.946700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:35:48.946700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:35:48.946700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:35:48.946700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:35:48.946700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:35:48.946700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 30 00:35:48.946700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:48.946700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:35:48.946700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:48.946700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:48.946700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:48.946700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:48.946700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:48.946700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:48.948937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:48.948937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:48.948937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:48.948937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:48.948937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:48.948937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:48.948937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:35:48.948937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:35:48.948937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:35:48.948937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:35:48.948937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:48.948937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:48.948937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:48.948937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:48.948937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:48.948937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:48.950594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:48.950594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:48.950594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:48.950594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:48.950594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:48.950594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:48.950594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:48.950594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:48.950594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:48.950594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:48.950594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:35:48.950594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:35:48.950594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 30 00:35:48.950594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:48.950594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 260/514] 10.222.0.36 () {56 vars in 1129 bytes} [Sun Oct 30 00:35:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:48.957265 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c66eaff-ad58-4edd-8b9a-bd8480fa35dd None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:48.993459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee163fb4-6ed7-4426-a2b0-eaa80336a23e None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:49.005219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:49.032985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ac8d5-b80c-42d2-80e0-8aa5650c3055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:49.034006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ac8d5-b80c-42d2-80e0-8aa5650c3055 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:49.034500 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ac8d5-b80c-42d2-80e0-8aa5650c3055 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:49.035013 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ac8d5-b80c-42d2-80e0-8aa5650c3055 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:49.035912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ac8d5-b80c-42d2-80e0-8aa5650c3055 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:49.036449 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ac8d5-b80c-42d2-80e0-8aa5650c3055 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:49.038108 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ac8d5-b80c-42d2-80e0-8aa5650c3055 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:49.048651 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 255/515] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:49.053091 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ac8d5-b80c-42d2-80e0-8aa5650c3055 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:49.080520 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c66eaff-ad58-4edd-8b9a-bd8480fa35dd None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:49.118423 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238d661-be23-4b39-84b0-c0cdd81787c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:49.119067 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238d661-be23-4b39-84b0-c0cdd81787c7 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:49.119874 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238d661-be23-4b39-84b0-c0cdd81787c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:49.120186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238d661-be23-4b39-84b0-c0cdd81787c7 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:49.120821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238d661-be23-4b39-84b0-c0cdd81787c7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:49.121123 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238d661-be23-4b39-84b0-c0cdd81787c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:49.122665 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238d661-be23-4b39-84b0-c0cdd81787c7 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:49.127377 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 261/516] 10.222.0.36 () {56 vars in 1124 bytes} [Sun Oct 30 00:35:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:49.128562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238d661-be23-4b39-84b0-c0cdd81787c7 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:49.144493 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:49.167481 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ac8d5-b80c-42d2-80e0-8aa5650c3055 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:49.203728 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e4c6e-9dbe-46de-a312-030f3f092160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:49.204676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e4c6e-9dbe-46de-a312-030f3f092160 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:49.205101 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e4c6e-9dbe-46de-a312-030f3f092160 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:49.205519 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e4c6e-9dbe-46de-a312-030f3f092160 None admin] PATH_INFO: `/v3/services/network` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:49.206244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e4c6e-9dbe-46de-a312-030f3f092160 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:49.207776 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e4c6e-9dbe-46de-a312-030f3f092160 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:49.222480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-be7e4c6e-9dbe-46de-a312-030f3f092160 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 30 00:35:49.222480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:49.222480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:35:49.222480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:35:49.222480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:35:49.222480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:35:49.222480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:35:49.222480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:35:49.222480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 30 00:35:49.222480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:49.222480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:35:49.222480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:49.222480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:49.222480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:49.222480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:49.222480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:49.222480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:49.223737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:49.223737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:49.223737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:49.223737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:49.223737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:49.223737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:49.223737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:35:49.223737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:35:49.223737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:35:49.223737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:35:49.223737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:49.223737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:49.223737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:49.223737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:49.223737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:49.223737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:49.224690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:49.224690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:49.224690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:49.224690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:49.224690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:49.224690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:49.224690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:49.224690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:49.224690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:49.224690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:49.224690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:35:49.224690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:35:49.224690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 30 00:35:49.224690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:49.227385 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 256/517] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:49 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:49.228481 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e4c6e-9dbe-46de-a312-030f3f092160 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:49.241921 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:49.246677 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238d661-be23-4b39-84b0-c0cdd81787c7 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:49.286852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aae82a-08f4-4970-af68-bb35f4677075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:49.287525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aae82a-08f4-4970-af68-bb35f4677075 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:49.287802 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aae82a-08f4-4970-af68-bb35f4677075 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:49.288073 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aae82a-08f4-4970-af68-bb35f4677075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:49.303052 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aae82a-08f4-4970-af68-bb35f4677075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:49.305098 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aae82a-08f4-4970-af68-bb35f4677075 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:49.324209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-92aae82a-08f4-4970-af68-bb35f4677075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:49.324209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:49.324209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:49.324209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:49.324209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:49.324209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:49.324209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:49.324209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:49.324209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:49.324209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:49.324209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:49.324209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:49.324209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:49.324209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:49.324209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:49.324209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:49.332132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:49.332132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:49.332132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:49.332132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:49.332132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:49.332132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:49.332132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:49.332132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:49.332132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:49.332132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:49.332132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:49.332132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:49.332132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:49.332132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:49.332132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:49.332132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:49.332132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:49.333219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:49.333219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:49.333219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:49.333219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 262/518] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:49.333219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aae82a-08f4-4970-af68-bb35f4677075 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:49.371560 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e4c6e-9dbe-46de-a312-030f3f092160 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:49.422868 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bc74e-1664-4fc1-ad05-6d4b03175483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:49.423489 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bc74e-1664-4fc1-ad05-6d4b03175483 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:49.423768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bc74e-1664-4fc1-ad05-6d4b03175483 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:49.424039 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6bc74e-1664-4fc1-ad05-6d4b03175483 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:49.424699 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6bc74e-1664-4fc1-ad05-6d4b03175483 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:49.424996 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6bc74e-1664-4fc1-ad05-6d4b03175483 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:49.426680 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6bc74e-1664-4fc1-ad05-6d4b03175483 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:49.436250 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 257/519] 10.222.0.36 () {56 vars in 1124 bytes} [Sun Oct 30 00:35:49 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:49.437538 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bc74e-1664-4fc1-ad05-6d4b03175483 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:49.466182 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:49.470023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aae82a-08f4-4970-af68-bb35f4677075 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:49.553158 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bfb53-4b60-497e-8f6f-f44b6d7117ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:49.554837 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bfb53-4b60-497e-8f6f-f44b6d7117ae None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:49.555117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bfb53-4b60-497e-8f6f-f44b6d7117ae None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:49.555388 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bfb53-4b60-497e-8f6f-f44b6d7117ae None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:49.556045 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bfb53-4b60-497e-8f6f-f44b6d7117ae None admin] RBAC: Adding query filter params (name=network) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:49.556372 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bfb53-4b60-497e-8f6f-f44b6d7117ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:49.557875 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bfb53-4b60-497e-8f6f-f44b6d7117ae None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:49.574050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 263/520] 10.222.0.36 () {56 vars in 1112 bytes} [Sun Oct 30 00:35:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:49.579154 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bfb53-4b60-497e-8f6f-f44b6d7117ae None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:49.620875 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:49.674490 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6bc74e-1664-4fc1-ad05-6d4b03175483 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:49.725983 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cfb8f-b3ce-412a-850e-3ae8b2d5839b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:49.727510 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cfb8f-b3ce-412a-850e-3ae8b2d5839b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:49.728007 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cfb8f-b3ce-412a-850e-3ae8b2d5839b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:49.728528 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cfb8f-b3ce-412a-850e-3ae8b2d5839b None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:49.730602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cfb8f-b3ce-412a-850e-3ae8b2d5839b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:49.731129 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cfb8f-b3ce-412a-850e-3ae8b2d5839b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:49.733368 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cfb8f-b3ce-412a-850e-3ae8b2d5839b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:49.749625 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 258/521] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:49.755365 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cfb8f-b3ce-412a-850e-3ae8b2d5839b None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:49.763019 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bfb53-4b60-497e-8f6f-f44b6d7117ae None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:49.823693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc025be7-d7be-48dd-8b23-5180a39edff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:49.824259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc025be7-d7be-48dd-8b23-5180a39edff0 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:49.824541 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc025be7-d7be-48dd-8b23-5180a39edff0 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:49.824815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc025be7-d7be-48dd-8b23-5180a39edff0 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:49.825774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc025be7-d7be-48dd-8b23-5180a39edff0 None admin] RBAC: Adding query filter params (type=network) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:49.826085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc025be7-d7be-48dd-8b23-5180a39edff0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:49.827491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc025be7-d7be-48dd-8b23-5180a39edff0 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:49.841803 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 264/522] 10.222.0.36 () {56 vars in 1112 bytes} [Sun Oct 30 00:35:49 2022] GET /identity/v3/services?type=network => generated 124 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:49.874214 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:49.958343 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cfb8f-b3ce-412a-850e-3ae8b2d5839b None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:50.018797 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66212390-65f9-4025-88a1-69dfe7f31303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:50.019535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66212390-65f9-4025-88a1-69dfe7f31303 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:50.019814 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66212390-65f9-4025-88a1-69dfe7f31303 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:50.020106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66212390-65f9-4025-88a1-69dfe7f31303 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:50.041437 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66212390-65f9-4025-88a1-69dfe7f31303 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:50.044144 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66212390-65f9-4025-88a1-69dfe7f31303 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:50.068256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-66212390-65f9-4025-88a1-69dfe7f31303 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:35:50.068256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:50.068256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:50.068256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:50.068256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:50.068256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:50.068256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:50.068256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:50.068256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:50.068256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:50.068256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:50.068256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:50.068256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:35:50.068256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:35:50.068256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:35:50.068256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:35:50.069270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:50.069270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:50.069270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:35:50.069270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:50.069270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:35:50.069270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:50.069270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:50.069270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:50.069270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:50.069270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:50.069270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:50.069270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:50.069270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:50.069270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:50.069270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:50.069270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:50.069270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:50.069270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:50.070210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:50.070210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:50.070210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:35:50.070210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:35:50.070210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:35:50.070210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:35:50.070210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:35:50.070210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:35:50.070210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:35:50.070210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:50.070210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 259/523] 10.222.0.36 () {56 vars in 1093 bytes} [Sun Oct 30 00:35:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:50.077759 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc025be7-d7be-48dd-8b23-5180a39edff0 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:50.172751 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:50.197188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc025be7-d7be-48dd-8b23-5180a39edff0 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:50.236380 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852f82d-e4d2-4e9b-8329-e5dcabbed971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:50.237415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852f82d-e4d2-4e9b-8329-e5dcabbed971 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:50.237935 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852f82d-e4d2-4e9b-8329-e5dcabbed971 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:50.238399 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852f82d-e4d2-4e9b-8329-e5dcabbed971 None admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:50.239432 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852f82d-e4d2-4e9b-8329-e5dcabbed971 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:50.239935 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852f82d-e4d2-4e9b-8329-e5dcabbed971 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:50.241731 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852f82d-e4d2-4e9b-8329-e5dcabbed971 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:50.267437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 265/524] 10.222.0.36 () {56 vars in 1133 bytes} [Sun Oct 30 00:35:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:50.274215 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66212390-65f9-4025-88a1-69dfe7f31303 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:50.411716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66212390-65f9-4025-88a1-69dfe7f31303 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:50.438244 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8210f-388a-4cb8-9009-b069aa93bce1 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:50.438599 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8210f-388a-4cb8-9009-b069aa93bce1 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:50.438795 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8210f-388a-4cb8-9009-b069aa93bce1 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:50.440166 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 266/525] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:35:50.449991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd6737-4ce3-4a75-b9ad-13bf19dd6539 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:50.450433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd6737-4ce3-4a75-b9ad-13bf19dd6539 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:50.450875 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd6737-4ce3-4a75-b9ad-13bf19dd6539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:50.468360 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:50.482272 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-52dd6737-4ce3-4a75-b9ad-13bf19dd6539 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:50.483800 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-52dd6737-4ce3-4a75-b9ad-13bf19dd6539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:50.498612 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3089efd-0711-4a6a-94fe-2d8fae3e97d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:50.499615 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3089efd-0711-4a6a-94fe-2d8fae3e97d6 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:50.500065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3089efd-0711-4a6a-94fe-2d8fae3e97d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:50.500547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3089efd-0711-4a6a-94fe-2d8fae3e97d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:50.524780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3089efd-0711-4a6a-94fe-2d8fae3e97d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:50.526840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3089efd-0711-4a6a-94fe-2d8fae3e97d6 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:50.542961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-c3089efd-0711-4a6a-94fe-2d8fae3e97d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:50.542961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:50.542961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:50.542961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:50.542961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:50.542961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:50.542961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:50.542961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:50.542961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:50.542961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:50.542961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:50.542961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:50.542961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:50.542961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:50.542961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:50.542961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:50.543990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:50.543990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:50.543990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:50.543990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:50.543990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:50.543990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:50.543990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:50.543990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:50.543990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:50.543990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:50.543990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:50.543990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:50.543990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:50.543990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:50.543990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:50.543990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:50.543990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:50.544944 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:50.544944 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:50.544944 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:50.544944 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 260/526] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:50.551800 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3089efd-0711-4a6a-94fe-2d8fae3e97d6 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:50.617251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 267/527] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:50.726178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3089efd-0711-4a6a-94fe-2d8fae3e97d6 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:50.784645 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33011a10-712b-4776-a9f8-000fe04c3ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:50.785276 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33011a10-712b-4776-a9f8-000fe04c3ee2 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:50.785559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33011a10-712b-4776-a9f8-000fe04c3ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:50.785831 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33011a10-712b-4776-a9f8-000fe04c3ee2 None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:50.786489 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33011a10-712b-4776-a9f8-000fe04c3ee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:50.786793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33011a10-712b-4776-a9f8-000fe04c3ee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:50.788208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33011a10-712b-4776-a9f8-000fe04c3ee2 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:50.808869 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 261/528] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:50.814741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd6737-4ce3-4a75-b9ad-13bf19dd6539 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:50.946220 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:50.995781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd6737-4ce3-4a75-b9ad-13bf19dd6539 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:51.047649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa097c0-7fde-44f9-a4a2-20a8b0ff531e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:51.048024 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc96027-2acb-4db2-b273-05635a87f5f7 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:51.048318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa097c0-7fde-44f9-a4a2-20a8b0ff531e None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:51.048422 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc96027-2acb-4db2-b273-05635a87f5f7 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:51.048600 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa097c0-7fde-44f9-a4a2-20a8b0ff531e None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:51.048767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc96027-2acb-4db2-b273-05635a87f5f7 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:51.048858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa097c0-7fde-44f9-a4a2-20a8b0ff531e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:51.050506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 262/529] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:51.056217 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5a06b-48c8-42d4-8749-50d7febd440d None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:51.056524 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5a06b-48c8-42d4-8749-50d7febd440d None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:51.056778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5a06b-48c8-42d4-8749-50d7febd440d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:51.064958 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa097c0-7fde-44f9-a4a2-20a8b0ff531e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:51.066462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa097c0-7fde-44f9-a4a2-20a8b0ff531e None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:51.078135 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-60b5a06b-48c8-42d4-8749-50d7febd440d None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:51.079366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-60b5a06b-48c8-42d4-8749-50d7febd440d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:51.079791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-3aa097c0-7fde-44f9-a4a2-20a8b0ff531e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:51.079791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:51.079791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:51.079791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:51.079791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:51.079791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:51.079791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:51.079791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:51.079791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:51.079791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:51.079791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:51.079791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:51.079791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:35:51.079791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:35:51.079791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:35:51.080998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:35:51.080998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:51.080998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:51.080998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:51.080998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:51.080998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:51.080998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:51.080998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:51.080998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:51.080998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:51.080998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:51.080998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:51.080998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:51.080998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:51.080998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:51.080998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:51.080998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:51.081878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:35:51.081878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:35:51.081878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:35:51.081878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:35:51.081878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:35:51.081878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:35:51.081878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:51.081878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:51.081878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 268/530] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:51.082371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa097c0-7fde-44f9-a4a2-20a8b0ff531e None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:51.125069 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:51.180116 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 263/531] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 30 00:35:51.181101 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b5a06b-48c8-42d4-8749-50d7febd440d None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:51.268634 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa097c0-7fde-44f9-a4a2-20a8b0ff531e None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:51.318334 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:51.334943 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fd191-e361-42db-9858-fa55c2f8894b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:51.336713 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fd191-e361-42db-9858-fa55c2f8894b None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:51.337015 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fd191-e361-42db-9858-fa55c2f8894b None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:51.337292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fd191-e361-42db-9858-fa55c2f8894b None admin] PATH_INFO: `/v3/roles/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:51.360575 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542fd191-e361-42db-9858-fa55c2f8894b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:51.369138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-542fd191-e361-42db-9858-fa55c2f8894b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:51.369138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:51.369138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:51.369138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:51.369138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:51.369138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:51.369138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:51.369138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:51.369138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:51.369138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:51.369138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:51.369138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:51.369138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:51.369138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:51.369138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:51.369138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:51.370462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:51.370462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:51.370462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:51.370462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:51.370462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:51.370462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:51.370462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:51.370462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:51.370462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:51.370462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:51.370462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:51.370462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:51.370462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:51.370462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:51.370462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:51.370462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:51.371303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:51.371303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:51.371303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:51.371303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:51.371303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:51.371303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:51.371303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:51.371303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:51.371303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:51.371303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:51.371303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:51.371303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:51.371303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 269/532] 10.222.0.36 () {56 vars in 1102 bytes} [Sun Oct 30 00:35:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:51.379094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fd191-e361-42db-9858-fa55c2f8894b None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:51.389012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b5a06b-48c8-42d4-8749-50d7febd440d None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:51.446542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332d41d-0517-4202-aeb1-a9e6f57ea74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:51.447159 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332d41d-0517-4202-aeb1-a9e6f57ea74a None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:51.447436 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332d41d-0517-4202-aeb1-a9e6f57ea74a None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:51.447740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332d41d-0517-4202-aeb1-a9e6f57ea74a None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:51.449135 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332d41d-0517-4202-aeb1-a9e6f57ea74a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:51.449442 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332d41d-0517-4202-aeb1-a9e6f57ea74a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:51.451176 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332d41d-0517-4202-aeb1-a9e6f57ea74a None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:51.482203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 264/533] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:35:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:51.485226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332d41d-0517-4202-aeb1-a9e6f57ea74a None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:51.544242 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:51.568961 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fd191-e361-42db-9858-fa55c2f8894b None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:51.615663 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332d41d-0517-4202-aeb1-a9e6f57ea74a None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:51.630433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4385973-9729-4130-a3f1-3b6d212845a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:51.632152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4385973-9729-4130-a3f1-3b6d212845a6 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:51.633098 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4385973-9729-4130-a3f1-3b6d212845a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:51.633668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4385973-9729-4130-a3f1-3b6d212845a6 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:51.635861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4385973-9729-4130-a3f1-3b6d212845a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:51.637779 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4385973-9729-4130-a3f1-3b6d212845a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:51.640030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4385973-9729-4130-a3f1-3b6d212845a6 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:51.656171 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 270/534] 10.222.0.36 () {56 vars in 1103 bytes} [Sun Oct 30 00:35:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:51.658000 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4385973-9729-4130-a3f1-3b6d212845a6 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:51.672136 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d0662-6c11-48cc-ac33-49dcbdf457e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:51.672888 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d0662-6c11-48cc-ac33-49dcbdf457e9 None admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:51.673163 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d0662-6c11-48cc-ac33-49dcbdf457e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:51.673484 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d0662-6c11-48cc-ac33-49dcbdf457e9 None admin] PATH_INFO: `/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/adbba7be39a64230bab160ce77ce7586/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:51.675507 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d0662-6c11-48cc-ac33-49dcbdf457e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab3a0e5fc5a344739b58ec8313639496, user_id=adbba7be39a64230bab160ce77ce7586, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:51.677048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d0662-6c11-48cc-ac33-49dcbdf457e9 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:51.698799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 265/535] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:35:51 2022] PUT /identity/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/adbba7be39a64230bab160ce77ce7586/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:35:51.700149 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d0662-6c11-48cc-ac33-49dcbdf457e9 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:51.734875 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:51.815261 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4385973-9729-4130-a3f1-3b6d212845a6 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:51.832805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:51.847580 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d0662-6c11-48cc-ac33-49dcbdf457e9 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:51.894066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c50097-2233-4169-b720-e46553463ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:51.895939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c50097-2233-4169-b720-e46553463ff3 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:51.895939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c50097-2233-4169-b720-e46553463ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:51.895939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c50097-2233-4169-b720-e46553463ff3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:51.909759 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c50097-2233-4169-b720-e46553463ff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:51.911650 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c50097-2233-4169-b720-e46553463ff3 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:51.913135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d14d0-0f1b-48ad-9e47-8529999589d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:51.914387 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d14d0-0f1b-48ad-9e47-8529999589d3 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:51.915530 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d14d0-0f1b-48ad-9e47-8529999589d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:51.916838 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d14d0-0f1b-48ad-9e47-8529999589d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:51.930924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-51c50097-2233-4169-b720-e46553463ff3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:51.930924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:51.930924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:51.930924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:51.930924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:51.930924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:51.930924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:51.930924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:51.930924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:51.930924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:51.930924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:51.930924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:51.930924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:51.930924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:51.930924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:51.930924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:51.932582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:51.932582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:51.932582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:51.932582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:51.932582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:51.932582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:51.932582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:51.932582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:51.932582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:51.932582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:51.932582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:51.932582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:51.932582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:51.932582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:51.932582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:51.932582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:51.932582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:51.933752 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:51.933752 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:51.933752 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:51.933752 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 266/536] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:51.941188 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c50097-2233-4169-b720-e46553463ff3 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:51.960663 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d14d0-0f1b-48ad-9e47-8529999589d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:51.968203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-cb1d14d0-0f1b-48ad-9e47-8529999589d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:35:51.968203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:51.968203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:51.968203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:51.968203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:51.968203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:51.968203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:51.968203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:51.968203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:51.968203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:51.968203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:51.968203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:51.968203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:51.968203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:51.968203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:51.968203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:51.969188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:51.969188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:51.969188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:51.969188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:51.969188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:51.969188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:51.969188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:51.969188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:51.969188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:51.969188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:51.969188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:51.969188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:51.969188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:51.969188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:51.969188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:51.969188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:51.970030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:51.970030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:51.970030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:51.970030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:51.970030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:51.970030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:51.970030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:51.970030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:51.970030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:51.970030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:51.970030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:35:51.970030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:51.970030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 271/537] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:35:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:51.975438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d14d0-0f1b-48ad-9e47-8529999589d3 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:52.131376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c50097-2233-4169-b720-e46553463ff3 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:52.155357 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d14d0-0f1b-48ad-9e47-8529999589d3 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:52.167798 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:52.184343 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa823d09-6acd-4fe2-a8bd-b0f8ede15388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:52.184970 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa823d09-6acd-4fe2-a8bd-b0f8ede15388 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:52.185295 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa823d09-6acd-4fe2-a8bd-b0f8ede15388 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:52.185582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa823d09-6acd-4fe2-a8bd-b0f8ede15388 None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:52.186352 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa823d09-6acd-4fe2-a8bd-b0f8ede15388 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:52.186762 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa823d09-6acd-4fe2-a8bd-b0f8ede15388 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:52.188198 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa823d09-6acd-4fe2-a8bd-b0f8ede15388 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:52.197144 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c025d4-2f36-4bb9-b864-0120dd931efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:52.198313 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c025d4-2f36-4bb9-b864-0120dd931efe None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:52.198740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c025d4-2f36-4bb9-b864-0120dd931efe None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:52.198847 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c025d4-2f36-4bb9-b864-0120dd931efe None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:52.199517 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c025d4-2f36-4bb9-b864-0120dd931efe None admin] RBAC: Adding query filter params (name=admin) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:52.199815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c025d4-2f36-4bb9-b864-0120dd931efe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:52.201261 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c025d4-2f36-4bb9-b864-0120dd931efe None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:52.203649 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 267/538] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:52.204974 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c8086-ebe4-4497-a8da-c89df250323e None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:52.205256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c8086-ebe4-4497-a8da-c89df250323e None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:52.206045 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c8086-ebe4-4497-a8da-c89df250323e None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:52.207102 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 268/539] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:52.211099 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 272/540] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:52.212194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c8086-ebe4-4497-a8da-c89df250323e None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:52.217596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf84d3-0897-4592-98d2-dda2f393c664 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:52.217898 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf84d3-0897-4592-98d2-dda2f393c664 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:52.218151 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf84d3-0897-4592-98d2-dda2f393c664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:52.219112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:52.239410 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-cbcf84d3-0897-4592-98d2-dda2f393c664 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:52.240639 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-cbcf84d3-0897-4592-98d2-dda2f393c664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:52.342942 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 273/541] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:52.343941 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf84d3-0897-4592-98d2-dda2f393c664 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:52.345987 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c8086-ebe4-4497-a8da-c89df250323e None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:52.391610 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafc553-042e-49fd-8e65-6bead1761f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:52.392192 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafc553-042e-49fd-8e65-6bead1761f5d None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:52.392468 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafc553-042e-49fd-8e65-6bead1761f5d None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:52.393369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafc553-042e-49fd-8e65-6bead1761f5d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:52.403147 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafc553-042e-49fd-8e65-6bead1761f5d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:52.404744 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafc553-042e-49fd-8e65-6bead1761f5d None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:52.420374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-ddafc553-042e-49fd-8e65-6bead1761f5d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 30 00:35:52.420374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:52.420374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:52.420374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:52.420374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:52.420374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:52.420374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:52.420374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:52.420374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:52.420374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:52.420374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:52.420374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:52.420374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:35:52.420374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:35:52.420374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:35:52.420374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:35:52.422571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:52.422571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:52.422571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:35:52.422571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:52.422571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:35:52.422571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:52.422571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:52.422571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:52.422571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:52.422571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:52.422571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:52.422571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:52.422571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:52.422571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:52.422571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:52.422571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:52.422571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:52.422571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:52.424815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:52.424815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:52.424815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:35:52.424815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:35:52.424815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:35:52.424815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:35:52.424815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:35:52.424815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:35:52.424815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 30 00:35:52.424815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:52.424815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 269/542] 10.222.0.36 () {56 vars in 1117 bytes} [Sun Oct 30 00:35:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:52.429916 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafc553-042e-49fd-8e65-6bead1761f5d None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:52.511088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:52.560419 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafc553-042e-49fd-8e65-6bead1761f5d None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:52.601324 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503228b-ebb1-4538-880c-95c1c9a81d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:52.601941 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503228b-ebb1-4538-880c-95c1c9a81d7a None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:52.602223 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503228b-ebb1-4538-880c-95c1c9a81d7a None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:52.602495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503228b-ebb1-4538-880c-95c1c9a81d7a None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:52.603236 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503228b-ebb1-4538-880c-95c1c9a81d7a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:52.603542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503228b-ebb1-4538-880c-95c1c9a81d7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:52.606035 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503228b-ebb1-4538-880c-95c1c9a81d7a None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:52.613098 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf84d3-0897-4592-98d2-dda2f393c664 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:52.614968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:52.648666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 270/543] 10.222.0.36 () {56 vars in 1149 bytes} [Sun Oct 30 00:35:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:52.655153 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503228b-ebb1-4538-880c-95c1c9a81d7a None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:52.690536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096b5df-bd03-4437-b6e3-b2b05fdc2023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:52.691194 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096b5df-bd03-4437-b6e3-b2b05fdc2023 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:52.691477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096b5df-bd03-4437-b6e3-b2b05fdc2023 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:52.691752 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096b5df-bd03-4437-b6e3-b2b05fdc2023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:52.717952 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096b5df-bd03-4437-b6e3-b2b05fdc2023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:52.719956 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096b5df-bd03-4437-b6e3-b2b05fdc2023 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:52.741864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-c096b5df-bd03-4437-b6e3-b2b05fdc2023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:52.741864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:52.741864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:52.741864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:52.741864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:52.741864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:52.741864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:52.741864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:52.741864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:52.741864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:52.741864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:52.741864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:52.741864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:52.741864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:52.741864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:52.741864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:52.742878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:52.742878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:52.742878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:52.742878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:52.742878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:52.742878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:52.742878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:52.742878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:52.742878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:52.742878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:52.742878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:52.742878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:52.742878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:52.742878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:52.742878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:52.742878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:52.742878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:52.743758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:52.743758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:52.743758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:52.743758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 274/544] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:52.744519 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096b5df-bd03-4437-b6e3-b2b05fdc2023 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:52.817786 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503228b-ebb1-4538-880c-95c1c9a81d7a None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:52.842868 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:52.859019 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893d838-3b24-4df5-844b-d76ac4349d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:52.859667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893d838-3b24-4df5-844b-d76ac4349d08 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:52.859946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893d838-3b24-4df5-844b-d76ac4349d08 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:52.860267 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893d838-3b24-4df5-844b-d76ac4349d08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:52.876184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893d838-3b24-4df5-844b-d76ac4349d08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:52.877521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893d838-3b24-4df5-844b-d76ac4349d08 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:52.899761 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096b5df-bd03-4437-b6e3-b2b05fdc2023 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:52.901293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-e893d838-3b24-4df5-844b-d76ac4349d08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:52.901293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:52.901293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:52.901293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:52.901293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:52.901293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:52.901293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:52.901293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:52.901293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:52.901293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:52.901293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:52.901293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:52.901293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:52.901293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:52.901293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:52.901293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:52.902287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:52.902287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:52.902287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:52.902287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:52.902287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:52.902287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:52.902287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:52.902287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:52.902287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:52.902287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:52.902287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:52.902287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:52.902287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:52.902287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:52.902287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:52.902287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:52.902287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:52.903203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:52.903203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:52.903203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:52.903203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 271/545] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:52.903431 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893d838-3b24-4df5-844b-d76ac4349d08 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:52.948109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d5023-0797-42ca-9cb4-3b20d28f3c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:52.948781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d5023-0797-42ca-9cb4-3b20d28f3c5f None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:52.949064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d5023-0797-42ca-9cb4-3b20d28f3c5f None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:52.949340 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d5023-0797-42ca-9cb4-3b20d28f3c5f None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:52.950003 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d5023-0797-42ca-9cb4-3b20d28f3c5f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:52.951403 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d5023-0797-42ca-9cb4-3b20d28f3c5f None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:52.971632 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 275/546] 10.222.0.36 () {60 vars in 1141 bytes} [Sun Oct 30 00:35:52 2022] POST /identity/v3/services => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:35:52.973305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539e2cc-83bb-4c81-b967-af3ee77971ef None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:52.973595 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539e2cc-83bb-4c81-b967-af3ee77971ef None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:52.973872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539e2cc-83bb-4c81-b967-af3ee77971ef None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:52.975275 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 276/547] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:52.981037 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539e2cc-83bb-4c81-b967-af3ee77971ef None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:53.007344 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:53.061430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893d838-3b24-4df5-844b-d76ac4349d08 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:53.120665 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895d3ea-d1de-447f-820a-b00e51eb4276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:53.121347 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895d3ea-d1de-447f-820a-b00e51eb4276 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:53.122214 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895d3ea-d1de-447f-820a-b00e51eb4276 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:53.122214 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895d3ea-d1de-447f-820a-b00e51eb4276 None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:53.122615 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895d3ea-d1de-447f-820a-b00e51eb4276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:53.122920 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895d3ea-d1de-447f-820a-b00e51eb4276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:53.124363 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895d3ea-d1de-447f-820a-b00e51eb4276 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:53.143593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 272/548] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:53.145870 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bde87a-f9a4-472e-b4d7-134247f88179 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:53.146316 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bde87a-f9a4-472e-b4d7-134247f88179 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:53.146768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bde87a-f9a4-472e-b4d7-134247f88179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:53.164531 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539e2cc-83bb-4c81-b967-af3ee77971ef None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:53.168364 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-05bde87a-f9a4-472e-b4d7-134247f88179 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:53.170371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-05bde87a-f9a4-472e-b4d7-134247f88179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:53.209011 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb0121-ebd1-4ecb-a611-1c6ee6ceb6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:53.209464 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb0121-ebd1-4ecb-a611-1c6ee6ceb6b7 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:53.209641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb0121-ebd1-4ecb-a611-1c6ee6ceb6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:53.209919 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb0121-ebd1-4ecb-a611-1c6ee6ceb6b7 None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:53.210920 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb0121-ebd1-4ecb-a611-1c6ee6ceb6b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:53.210920 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb0121-ebd1-4ecb-a611-1c6ee6ceb6b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:53.213035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb0121-ebd1-4ecb-a611-1c6ee6ceb6b7 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:53.213383 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:53.240837 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 277/549] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:53.241763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb0121-ebd1-4ecb-a611-1c6ee6ceb6b7 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:53.277237 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 273/550] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:53 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 30 00:35:53.278220 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bde87a-f9a4-472e-b4d7-134247f88179 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:53.365026 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:53.430995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb0121-ebd1-4ecb-a611-1c6ee6ceb6b7 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:53.448311 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bde87a-f9a4-472e-b4d7-134247f88179 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:53.507975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06aa28-1877-4808-a171-37187049d37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:53.508653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06aa28-1877-4808-a171-37187049d37a None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:53.510684 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06aa28-1877-4808-a171-37187049d37a None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:53.510969 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06aa28-1877-4808-a171-37187049d37a None admin] PATH_INFO: `/v3/users/swift` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:53.527891 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c12997-8015-45ee-844c-afc3cab6012f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:53.528570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c12997-8015-45ee-844c-afc3cab6012f None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:53.528850 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c12997-8015-45ee-844c-afc3cab6012f None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:53.529679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06aa28-1877-4808-a171-37187049d37a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:53.531235 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06aa28-1877-4808-a171-37187049d37a None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:53.532023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c12997-8015-45ee-844c-afc3cab6012f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:53.546564 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c12997-8015-45ee-844c-afc3cab6012f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:53.548196 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c12997-8015-45ee-844c-afc3cab6012f None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:53.554066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-bb06aa28-1877-4808-a171-37187049d37a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 30 00:35:53.554066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:53.554066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:53.554066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:53.554066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:53.554066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:53.554066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:53.554066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:53.554066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:53.554066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:53.554066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:53.554066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:53.554066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:35:53.554066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:35:53.554066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:35:53.554066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:35:53.555023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:53.555023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:53.555023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:35:53.555023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:53.555023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:35:53.555023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:53.555023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:53.555023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:53.555023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:53.555023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:53.555023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:53.555023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:53.555023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:53.555023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:53.555023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:53.555023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:53.555023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:53.555023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:53.555950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:53.555950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:53.555950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:35:53.555950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:35:53.555950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:35:53.555950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:35:53.555950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:35:53.555950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:35:53.555950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 30 00:35:53.555950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:53.559626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 278/551] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:35:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:53.562841 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06aa28-1877-4808-a171-37187049d37a None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:53.574588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-75c12997-8015-45ee-844c-afc3cab6012f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:53.574588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:53.574588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:53.574588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:53.574588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:53.574588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:53.574588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:53.574588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:53.574588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:53.574588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:53.574588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:53.574588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:53.574588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:35:53.574588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:35:53.574588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:35:53.575542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:35:53.575542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:53.575542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:53.575542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:53.575542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:53.575542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:53.575542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:53.575542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:53.575542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:53.575542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:53.575542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:53.575542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:53.575542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:53.575542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:53.575542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:53.575542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:53.575542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:53.576464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:35:53.576464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:35:53.576464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:35:53.576464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:35:53.576464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:35:53.576464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:35:53.576464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:53.576464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:53.578887 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 274/552] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:53.589645 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c12997-8015-45ee-844c-afc3cab6012f None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:53.620519 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:53.730120 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c12997-8015-45ee-844c-afc3cab6012f None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:53.733411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06aa28-1877-4808-a171-37187049d37a None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:53.777371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4a6e3-913d-4718-ac87-c8bf1e40dfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:53.777371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4a6e3-913d-4718-ac87-c8bf1e40dfbc None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:53.777774 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4a6e3-913d-4718-ac87-c8bf1e40dfbc None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:53.778440 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4a6e3-913d-4718-ac87-c8bf1e40dfbc None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:53.779055 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4a6e3-913d-4718-ac87-c8bf1e40dfbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:53.779370 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4a6e3-913d-4718-ac87-c8bf1e40dfbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:53.780851 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4a6e3-913d-4718-ac87-c8bf1e40dfbc None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:53.787102 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d5d3d-f6ca-468b-98f5-d9e1d40e5f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:53.787736 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d5d3d-f6ca-468b-98f5-d9e1d40e5f96 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:53.788021 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d5d3d-f6ca-468b-98f5-d9e1d40e5f96 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:53.788323 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d5d3d-f6ca-468b-98f5-d9e1d40e5f96 None admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:53.789721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d5d3d-f6ca-468b-98f5-d9e1d40e5f96 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:53.790029 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d5d3d-f6ca-468b-98f5-d9e1d40e5f96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:53.792875 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d5d3d-f6ca-468b-98f5-d9e1d40e5f96 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:53.794772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:53.799818 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 275/553] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:35:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:53.800812 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4a6e3-913d-4718-ac87-c8bf1e40dfbc None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:53.828186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 279/554] 10.222.0.36 () {56 vars in 1135 bytes} [Sun Oct 30 00:35:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:53.829128 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d5d3d-f6ca-468b-98f5-d9e1d40e5f96 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:53.914415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:53.948310 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4a6e3-913d-4718-ac87-c8bf1e40dfbc None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:53.973276 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d5d3d-f6ca-468b-98f5-d9e1d40e5f96 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:53.995951 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0d2df-c069-4974-83df-c8b89a3339d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:53.997389 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0d2df-c069-4974-83df-c8b89a3339d1 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:53.998108 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0d2df-c069-4974-83df-c8b89a3339d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:53.998534 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0d2df-c069-4974-83df-c8b89a3339d1 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:53.999377 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0d2df-c069-4974-83df-c8b89a3339d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:54.001635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0d2df-c069-4974-83df-c8b89a3339d1 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:54.022203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710a1b5-7cd4-497a-830e-c7b7fee5361c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:54.023058 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710a1b5-7cd4-497a-830e-c7b7fee5361c None admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:54.023441 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710a1b5-7cd4-497a-830e-c7b7fee5361c None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:54.023815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710a1b5-7cd4-497a-830e-c7b7fee5361c None admin] PATH_INFO: `/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/8a91d3b9c4834d729211ec367e235889/roles/9a8de43206fa4a8fa4901488bf86f9f5` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:54.030630 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 276/555] 10.222.0.36 () {60 vars in 1141 bytes} [Sun Oct 30 00:35:53 2022] POST /identity/v3/services => generated 240 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:35:54.030630 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0d2df-c069-4974-83df-c8b89a3339d1 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:54.057178 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710a1b5-7cd4-497a-830e-c7b7fee5361c None admin] RBAC: Authorizing `identity:create_grant(project_id=ab3a0e5fc5a344739b58ec8313639496, user_id=8a91d3b9c4834d729211ec367e235889, role_id=9a8de43206fa4a8fa4901488bf86f9f5)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:54.059319 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710a1b5-7cd4-497a-830e-c7b7fee5361c None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:54.085266 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 280/556] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:35:53 2022] PUT /identity/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/8a91d3b9c4834d729211ec367e235889/roles/9a8de43206fa4a8fa4901488bf86f9f5 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:35:54.216881 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0d2df-c069-4974-83df-c8b89a3339d1 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:54.268136 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83900085-0c91-4884-8d8c-83970a5366b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:54.268729 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83900085-0c91-4884-8d8c-83970a5366b6 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:54.269010 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83900085-0c91-4884-8d8c-83970a5366b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:54.269285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83900085-0c91-4884-8d8c-83970a5366b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:54.271707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:54.284566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83900085-0c91-4884-8d8c-83970a5366b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:54.286502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83900085-0c91-4884-8d8c-83970a5366b6 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:54.304658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-83900085-0c91-4884-8d8c-83970a5366b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:54.304658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:54.304658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:54.304658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:54.304658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:54.304658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:54.304658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:54.304658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:54.304658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:54.304658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:54.304658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:54.304658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:54.304658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:54.304658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:54.304658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:54.304658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:54.305934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:54.305934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:54.305934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:54.305934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:54.305934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:54.305934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:54.305934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:54.305934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:54.305934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:54.305934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:54.305934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:54.305934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:54.305934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:54.305934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:54.305934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:54.305934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:54.305934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:54.307087 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:54.307087 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:54.307087 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:54.311430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 277/557] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:54.313130 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710a1b5-7cd4-497a-830e-c7b7fee5361c None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:54.346395 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cb07c-6023-4744-9291-f2d716e02a19 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:54.346718 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cb07c-6023-4744-9291-f2d716e02a19 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:54.346933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cb07c-6023-4744-9291-f2d716e02a19 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:54.348592 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 278/558] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:35:54.371575 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:54.373582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f5fe0-04e3-4bca-a732-c724761cc30f None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:54.373875 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f5fe0-04e3-4bca-a732-c724761cc30f None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:54.375751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f5fe0-04e3-4bca-a732-c724761cc30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:54.398795 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-009f5fe0-04e3-4bca-a732-c724761cc30f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:54.400431 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-009f5fe0-04e3-4bca-a732-c724761cc30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:54.464557 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710a1b5-7cd4-497a-830e-c7b7fee5361c None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:54.512695 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 279/559] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:35:54.527322 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb389d-07f7-48a5-98e9-de3f081c7119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:54.528276 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb389d-07f7-48a5-98e9-de3f081c7119 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:54.528700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb389d-07f7-48a5-98e9-de3f081c7119 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:54.529136 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb389d-07f7-48a5-98e9-de3f081c7119 None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:54.529929 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb389d-07f7-48a5-98e9-de3f081c7119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:54.530431 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb389d-07f7-48a5-98e9-de3f081c7119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:54.531985 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb389d-07f7-48a5-98e9-de3f081c7119 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:54.546388 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 281/560] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:54.553286 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f5fe0-04e3-4bca-a732-c724761cc30f None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:54.706408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f5fe0-04e3-4bca-a732-c724761cc30f None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:54.726145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:54.748875 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57643761-ae5b-4d5b-aaa2-9fbac90cce2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:54.749868 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57643761-ae5b-4d5b-aaa2-9fbac90cce2b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:54.750312 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57643761-ae5b-4d5b-aaa2-9fbac90cce2b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:54.750784 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57643761-ae5b-4d5b-aaa2-9fbac90cce2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:54.758462 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57643761-ae5b-4d5b-aaa2-9fbac90cce2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:54.759931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57643761-ae5b-4d5b-aaa2-9fbac90cce2b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:54.771862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-57643761-ae5b-4d5b-aaa2-9fbac90cce2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:54.771862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:54.771862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:54.771862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:54.771862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:54.771862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:54.771862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:54.771862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:54.771862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:54.771862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:54.771862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:54.771862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:54.771862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:35:54.771862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:35:54.771862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:35:54.772829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:35:54.772829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:54.772829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:54.772829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:54.772829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:54.772829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:54.772829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:54.772829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:54.772829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:54.772829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:54.772829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:54.772829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:54.772829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:54.772829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:54.772829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:54.772829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:54.772829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:54.773720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:35:54.773720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:35:54.773720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:35:54.773720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:35:54.773720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:35:54.773720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:35:54.773720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:54.773720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:54.777914 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 280/561] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:54.779784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb389d-07f7-48a5-98e9-de3f081c7119 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:54.884307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57643761-ae5b-4d5b-aaa2-9fbac90cce2b None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:54.951253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb389d-07f7-48a5-98e9-de3f081c7119 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:54.955701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:55.028596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947d850-3048-47d1-8856-ecf55fd5634f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:55.029237 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947d850-3048-47d1-8856-ecf55fd5634f None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:55.029520 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947d850-3048-47d1-8856-ecf55fd5634f None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:55.030621 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947d850-3048-47d1-8856-ecf55fd5634f None admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:55.031347 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947d850-3048-47d1-8856-ecf55fd5634f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:55.031657 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947d850-3048-47d1-8856-ecf55fd5634f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:55.033153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947d850-3048-47d1-8856-ecf55fd5634f None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:55.067594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 282/562] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:35:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:55.071699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947d850-3048-47d1-8856-ecf55fd5634f None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:55.097158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57643761-ae5b-4d5b-aaa2-9fbac90cce2b None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:55.171382 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bb8e5-4646-4b6a-9a43-9cd6b47a0319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:55.172063 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bb8e5-4646-4b6a-9a43-9cd6b47a0319 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:55.172369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bb8e5-4646-4b6a-9a43-9cd6b47a0319 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:55.172647 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bb8e5-4646-4b6a-9a43-9cd6b47a0319 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:55.185861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0bb8e5-4646-4b6a-9a43-9cd6b47a0319 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:55.191972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-9c0bb8e5-4646-4b6a-9a43-9cd6b47a0319 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:35:55.191972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:55.191972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:55.191972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:55.191972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:55.191972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:55.191972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:55.191972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:55.191972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:55.191972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:55.191972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:55.191972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:55.191972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:55.191972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:55.191972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:55.191972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:55.193065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:55.193065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:55.193065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:55.193065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:55.193065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:55.193065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:55.193065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:55.193065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:55.193065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:55.193065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:55.193065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:55.193065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:55.193065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:55.193065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:55.193065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:55.193065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:55.193927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:55.193927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:55.193927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:55.193927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:55.193927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:55.193927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:55.193927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:55.193927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:55.193927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:55.193927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:55.193927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:35:55.193927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:55.193927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 281/563] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:35:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:55.202546 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bb8e5-4646-4b6a-9a43-9cd6b47a0319 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:55.276823 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:55.295225 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947d850-3048-47d1-8856-ecf55fd5634f None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:55.365162 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55140567-31eb-432a-a6ca-99aac77495f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:55.365984 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55140567-31eb-432a-a6ca-99aac77495f9 None admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:55.366407 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55140567-31eb-432a-a6ca-99aac77495f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:55.366688 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55140567-31eb-432a-a6ca-99aac77495f9 None admin] PATH_INFO: `/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/9f4c779b8422432fb06cdf996841b9b5/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:55.371431 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55140567-31eb-432a-a6ca-99aac77495f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab3a0e5fc5a344739b58ec8313639496, user_id=9f4c779b8422432fb06cdf996841b9b5, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:55.373022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55140567-31eb-432a-a6ca-99aac77495f9 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:55.395586 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bb8e5-4646-4b6a-9a43-9cd6b47a0319 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:55.397322 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 283/564] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:35:55 2022] PUT /identity/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/9f4c779b8422432fb06cdf996841b9b5/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:35:55.451226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb11ab4-a585-4ee4-b198-7825377e6d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:55.451862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb11ab4-a585-4ee4-b198-7825377e6d7e None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:55.452158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb11ab4-a585-4ee4-b198-7825377e6d7e None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:55.452434 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb11ab4-a585-4ee4-b198-7825377e6d7e None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:55.453186 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb11ab4-a585-4ee4-b198-7825377e6d7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:55.453495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb11ab4-a585-4ee4-b198-7825377e6d7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:55.454981 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb11ab4-a585-4ee4-b198-7825377e6d7e None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:55.465390 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 282/565] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:55.473380 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55140567-31eb-432a-a6ca-99aac77495f9 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:55.509983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:55.615223 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55140567-31eb-432a-a6ca-99aac77495f9 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:55.655830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef2be7-6dca-4b45-a4ae-03e7a31140ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:55.656614 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef2be7-6dca-4b45-a4ae-03e7a31140ba None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:55.656892 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef2be7-6dca-4b45-a4ae-03e7a31140ba None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:55.657166 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef2be7-6dca-4b45-a4ae-03e7a31140ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:55.670572 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef2be7-6dca-4b45-a4ae-03e7a31140ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:55.672190 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef2be7-6dca-4b45-a4ae-03e7a31140ba None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:55.688401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-e3ef2be7-6dca-4b45-a4ae-03e7a31140ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:55.688401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:55.688401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:55.688401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:55.688401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:55.688401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:55.688401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:55.688401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:55.688401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:55.688401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:55.688401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:55.688401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:55.688401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:55.688401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:55.688401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:55.688401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:55.689433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:55.689433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:55.689433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:55.689433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:55.689433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:55.689433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:55.689433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:55.689433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:55.689433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:55.689433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:55.689433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:55.689433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:55.689433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:55.689433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:55.689433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:55.689433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:55.689433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:55.690327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:55.690327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:55.690327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:55.690327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 284/566] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:55.697489 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb11ab4-a585-4ee4-b198-7825377e6d7e None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:55.733435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9102d1f-d331-4622-9864-25487b165b4c None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:55.733783 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9102d1f-d331-4622-9864-25487b165b4c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:55.733984 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9102d1f-d331-4622-9864-25487b165b4c None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:55.735510 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 285/567] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:35:55.741107 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635fcbd-b878-4c98-9ec3-c297fb52c5db None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:55.741424 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635fcbd-b878-4c98-9ec3-c297fb52c5db None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:55.741649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635fcbd-b878-4c98-9ec3-c297fb52c5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:55.777250 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-5635fcbd-b878-4c98-9ec3-c297fb52c5db None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:55.778522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-5635fcbd-b878-4c98-9ec3-c297fb52c5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:55.857221 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb11ab4-a585-4ee4-b198-7825377e6d7e None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:55.859478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:55.880179 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 286/568] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:35:55.929276 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e54de3-490a-49a8-8e87-7e9c96477b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:55.929906 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e54de3-490a-49a8-8e87-7e9c96477b01 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:55.930180 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e54de3-490a-49a8-8e87-7e9c96477b01 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:55.930449 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e54de3-490a-49a8-8e87-7e9c96477b01 None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:55.931095 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e54de3-490a-49a8-8e87-7e9c96477b01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:55.931393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e54de3-490a-49a8-8e87-7e9c96477b01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:55.932842 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e54de3-490a-49a8-8e87-7e9c96477b01 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:55.953632 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 283/569] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:55.959607 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635fcbd-b878-4c98-9ec3-c297fb52c5db None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:55.973184 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:56.107987 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635fcbd-b878-4c98-9ec3-c297fb52c5db None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:56.161442 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b51f54-ddca-447a-8ba2-237123b241f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:56.162098 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b51f54-ddca-447a-8ba2-237123b241f9 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:56.162381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b51f54-ddca-447a-8ba2-237123b241f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:56.162658 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b51f54-ddca-447a-8ba2-237123b241f9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:56.170965 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b51f54-ddca-447a-8ba2-237123b241f9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:56.172500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b51f54-ddca-447a-8ba2-237123b241f9 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:56.185152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-e4b51f54-ddca-447a-8ba2-237123b241f9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:35:56.185152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:56.185152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:56.185152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:56.185152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:56.185152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:56.185152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:56.185152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:56.185152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:56.185152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:56.185152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:56.185152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:56.185152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:35:56.185152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:35:56.185152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:35:56.185152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:35:56.186181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:56.186181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:56.186181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:35:56.186181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:56.186181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:35:56.186181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:56.186181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:56.186181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:56.186181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:56.186181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:56.186181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:56.186181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:56.186181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:56.186181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:56.186181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:56.186181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:56.186181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:56.186181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:56.187264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:56.187264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:56.187264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:35:56.187264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:35:56.187264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:35:56.187264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:35:56.187264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:35:56.187264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:35:56.187264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:35:56.187264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:56.191058 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 287/570] 10.222.0.36 () {56 vars in 1093 bytes} [Sun Oct 30 00:35:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:56.192490 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e54de3-490a-49a8-8e87-7e9c96477b01 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:56.290397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b51f54-ddca-447a-8ba2-237123b241f9 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:56.344294 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e54de3-490a-49a8-8e87-7e9c96477b01 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:56.410004 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:56.416156 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d4f00-8517-448e-978b-e71ed7a8f71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:56.417247 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d4f00-8517-448e-978b-e71ed7a8f71b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:56.417516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d4f00-8517-448e-978b-e71ed7a8f71b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:56.417789 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d4f00-8517-448e-978b-e71ed7a8f71b None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:56.418732 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d4f00-8517-448e-978b-e71ed7a8f71b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:56.419044 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d4f00-8517-448e-978b-e71ed7a8f71b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:56.420536 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d4f00-8517-448e-978b-e71ed7a8f71b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:56.441637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 284/571] 10.222.0.36 () {56 vars in 1133 bytes} [Sun Oct 30 00:35:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:56.450921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d4f00-8517-448e-978b-e71ed7a8f71b None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:56.481960 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:56.510014 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b51f54-ddca-447a-8ba2-237123b241f9 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:56.586849 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d4f00-8517-448e-978b-e71ed7a8f71b None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:56.613314 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757bec4-fa81-4eab-94ea-8c967f6a65c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:56.613553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757bec4-fa81-4eab-94ea-8c967f6a65c5 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:56.613821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757bec4-fa81-4eab-94ea-8c967f6a65c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:56.614099 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757bec4-fa81-4eab-94ea-8c967f6a65c5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:56.614696 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1757bec4-fa81-4eab-94ea-8c967f6a65c5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:56.616112 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1757bec4-fa81-4eab-94ea-8c967f6a65c5 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:56.629672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-1757bec4-fa81-4eab-94ea-8c967f6a65c5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 30 00:35:56.629672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:56.629672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:35:56.629672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:35:56.629672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:35:56.629672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:35:56.629672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:35:56.629672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:35:56.629672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 30 00:35:56.629672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:56.629672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:35:56.629672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:56.629672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:56.629672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:56.629672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:56.629672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:56.629672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:56.630830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:56.630830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:56.630830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:56.630830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:56.630830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:56.630830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:56.630830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:35:56.630830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:35:56.630830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:35:56.630830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:35:56.630830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:56.630830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:56.630830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:56.630830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:56.630830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:56.630830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:56.637740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:56.637740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:56.637740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:56.637740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:56.637740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:56.637740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:56.637740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:56.637740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:56.637740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:56.637740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:56.637740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:35:56.637740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:35:56.637740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 30 00:35:56.637740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:56.637740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 288/572] 10.222.0.36 () {56 vars in 1129 bytes} [Sun Oct 30 00:35:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:35:56.640024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b91b5-3991-4924-b6f1-8ba8d91af305 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:56.640024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b91b5-3991-4924-b6f1-8ba8d91af305 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:56.640024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b91b5-3991-4924-b6f1-8ba8d91af305 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:56.640024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 289/573] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:56.640509 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483ddc8-4926-4153-bd6b-86ace9664208 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:56.640775 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483ddc8-4926-4153-bd6b-86ace9664208 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:56.641042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483ddc8-4926-4153-bd6b-86ace9664208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:56.643722 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb81482-8252-466b-840e-723cc3e849f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:56.644850 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb81482-8252-466b-840e-723cc3e849f0 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:56.645511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb81482-8252-466b-840e-723cc3e849f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:56.646006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb81482-8252-466b-840e-723cc3e849f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:56.662265 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-2483ddc8-4926-4153-bd6b-86ace9664208 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:56.664406 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-2483ddc8-4926-4153-bd6b-86ace9664208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:56.736296 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb81482-8252-466b-840e-723cc3e849f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:56.738304 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb81482-8252-466b-840e-723cc3e849f0 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:56.756580 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 290/574] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:35:56.757769 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483ddc8-4926-4153-bd6b-86ace9664208 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:56.759282 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:56.864831 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 285/575] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:35:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 30 00:35:56.866610 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb81482-8252-466b-840e-723cc3e849f0 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:56.887093 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:56.987249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483ddc8-4926-4153-bd6b-86ace9664208 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:57.000446 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb81482-8252-466b-840e-723cc3e849f0 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:57.020931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:57.049689 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309b6af-d832-4911-83c9-945c7a91aa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:57.050743 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309b6af-d832-4911-83c9-945c7a91aa6a None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:57.051023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309b6af-d832-4911-83c9-945c7a91aa6a None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:57.051292 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309b6af-d832-4911-83c9-945c7a91aa6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:57.062901 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ae557-1b5c-4511-8688-e35352bbff31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:57.063535 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ae557-1b5c-4511-8688-e35352bbff31 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:57.063812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ae557-1b5c-4511-8688-e35352bbff31 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:57.064099 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ae557-1b5c-4511-8688-e35352bbff31 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:57.064524 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309b6af-d832-4911-83c9-945c7a91aa6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:57.064736 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ae557-1b5c-4511-8688-e35352bbff31 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:57.065037 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ae557-1b5c-4511-8688-e35352bbff31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:57.066139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309b6af-d832-4911-83c9-945c7a91aa6a None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:57.066569 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ae557-1b5c-4511-8688-e35352bbff31 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:57.081569 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 291/576] 10.222.0.36 () {56 vars in 1124 bytes} [Sun Oct 30 00:35:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:57.085346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-b309b6af-d832-4911-83c9-945c7a91aa6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:57.085346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:57.085346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:57.085346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:57.085346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:57.085346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:57.085346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:57.085346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:57.085346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:57.085346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:57.085346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:57.085346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:57.085346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:57.085346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:57.085346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:57.085346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:57.086277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:57.086277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:57.086277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:57.086277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:57.086277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:57.086277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:57.086277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:57.086277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:57.086277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:57.086277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:57.086277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:57.086277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:57.086277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:57.086277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:57.086277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:57.086277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:57.086277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:57.087167 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:57.087167 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:57.087167 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:57.087337 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ae557-1b5c-4511-8688-e35352bbff31 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:57.094755 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 286/577] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:57.097144 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309b6af-d832-4911-83c9-945c7a91aa6a None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:57.221543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ae557-1b5c-4511-8688-e35352bbff31 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:57.225083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309b6af-d832-4911-83c9-945c7a91aa6a None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:57.252577 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:57.263177 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046dac7-4975-48fd-a66f-19368e2e776e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:57.263795 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046dac7-4975-48fd-a66f-19368e2e776e None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:57.264072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046dac7-4975-48fd-a66f-19368e2e776e None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:57.264366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046dac7-4975-48fd-a66f-19368e2e776e None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:57.265004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046dac7-4975-48fd-a66f-19368e2e776e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:57.265303 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046dac7-4975-48fd-a66f-19368e2e776e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:57.266241 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e71c2b-898a-4937-8ca1-119813f7361c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:57.266671 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046dac7-4975-48fd-a66f-19368e2e776e None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:57.266850 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e71c2b-898a-4937-8ca1-119813f7361c None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:57.267122 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e71c2b-898a-4937-8ca1-119813f7361c None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:57.267390 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e71c2b-898a-4937-8ca1-119813f7361c None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:57.268015 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e71c2b-898a-4937-8ca1-119813f7361c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:57.268586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e71c2b-898a-4937-8ca1-119813f7361c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:57.269961 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e71c2b-898a-4937-8ca1-119813f7361c None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:57.280668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 292/578] 10.222.0.36 () {56 vars in 1124 bytes} [Sun Oct 30 00:35:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:57.280668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7747700-77fc-4e0c-9fd6-b0110ae8e310 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:57.280668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7747700-77fc-4e0c-9fd6-b0110ae8e310 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:57.280668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7747700-77fc-4e0c-9fd6-b0110ae8e310 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:57.280668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 293/579] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:57.280668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7747700-77fc-4e0c-9fd6-b0110ae8e310 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:57.291258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 287/580] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:57.293471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046dac7-4975-48fd-a66f-19368e2e776e None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:57.331722 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:57.411315 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7747700-77fc-4e0c-9fd6-b0110ae8e310 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:57.419647 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046dac7-4975-48fd-a66f-19368e2e776e None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:57.449952 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96da0a0-a6df-48fc-9348-b350a130bc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:57.450596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96da0a0-a6df-48fc-9348-b350a130bc9c None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:57.450876 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96da0a0-a6df-48fc-9348-b350a130bc9c None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:57.451149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96da0a0-a6df-48fc-9348-b350a130bc9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:57.467811 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96da0a0-a6df-48fc-9348-b350a130bc9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:57.468034 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f02be-b89e-4821-96b7-8e79837ef8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:57.468620 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f02be-b89e-4821-96b7-8e79837ef8df None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:57.468900 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f02be-b89e-4821-96b7-8e79837ef8df None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:57.469171 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f02be-b89e-4821-96b7-8e79837ef8df None admin] PATH_INFO: `/v3/endpoints` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:57.469874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f02be-b89e-4821-96b7-8e79837ef8df None admin] RBAC: Adding query filter params (interface=public, service_id=7d3c2e0c95e0403e9abf40e97cf96169, region_id=RegionOne) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:57.470179 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f02be-b89e-4821-96b7-8e79837ef8df None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:57.471551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f02be-b89e-4821-96b7-8e79837ef8df None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:57.472734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-f96da0a0-a6df-48fc-9348-b350a130bc9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:57.472734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:57.472734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:57.472734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:57.472734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:57.472734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:57.472734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:57.472734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:57.472734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:57.472734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:57.472734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:57.472734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:57.472734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:57.472734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:57.472734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:57.472734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:57.473624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:57.473624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:57.473624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:57.473624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:57.473624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:57.473624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:57.473624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:57.473624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:57.473624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:57.473624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:57.473624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:57.473624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:57.473624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:57.473624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:57.473624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:57.473624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:57.474739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:57.474739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:57.474739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:57.474739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:57.474739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:57.474739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:57.474739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:57.474739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:57.474739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:57.474739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:57.474739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:35:57.474739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:57.474739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 294/581] 10.222.0.36 () {56 vars in 1102 bytes} [Sun Oct 30 00:35:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:57.476185 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfb970-04e0-4510-b5fb-3b168ab852c2 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:57.476185 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfb970-04e0-4510-b5fb-3b168ab852c2 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:57.476185 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfb970-04e0-4510-b5fb-3b168ab852c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:57.485992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 288/582] 10.222.0.36 () {56 vars in 1251 bytes} [Sun Oct 30 00:35:57 2022] GET /identity/v3/endpoints?service_id=7d3c2e0c95e0403e9abf40e97cf96169&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:57.491286 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f02be-b89e-4821-96b7-8e79837ef8df None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:57.495997 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-85bfb970-04e0-4510-b5fb-3b168ab852c2 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:57.497316 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-85bfb970-04e0-4510-b5fb-3b168ab852c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:57.544419 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:57.580120 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 295/583] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:35:57.580120 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfb970-04e0-4510-b5fb-3b168ab852c2 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:57.613617 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:57.630418 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f02be-b89e-4821-96b7-8e79837ef8df None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:57.685002 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96773b7b-94e2-4694-b7b7-3454ba2f42a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:57.685650 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96773b7b-94e2-4694-b7b7-3454ba2f42a6 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:57.685928 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96773b7b-94e2-4694-b7b7-3454ba2f42a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:57.686197 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96773b7b-94e2-4694-b7b7-3454ba2f42a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:57.695396 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96773b7b-94e2-4694-b7b7-3454ba2f42a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:57.696914 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96773b7b-94e2-4694-b7b7-3454ba2f42a6 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:57.710066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-96773b7b-94e2-4694-b7b7-3454ba2f42a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:57.710066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:57.710066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:57.710066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:57.710066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:57.710066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:57.710066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:57.710066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:57.710066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:57.710066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:57.710066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:57.710066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:57.710066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:35:57.710066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:35:57.710066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:35:57.711085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:35:57.711085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:57.711085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:57.711085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:57.711085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:57.711085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:57.711085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:57.711085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:57.711085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:57.711085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:57.711085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:57.711085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:57.711085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:57.711085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:57.711085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:57.711085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:57.711085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:57.711996 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:35:57.711996 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:35:57.711996 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:35:57.711996 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:35:57.711996 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:35:57.711996 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:35:57.711996 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:35:57.711996 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:57.712485 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfb970-04e0-4510-b5fb-3b168ab852c2 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:57.716859 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 289/584] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:57.717860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96773b7b-94e2-4694-b7b7-3454ba2f42a6 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:57.763180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7235d3e-869d-47ad-9fc9-2f7c131c8214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:57.763843 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7235d3e-869d-47ad-9fc9-2f7c131c8214 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:57.764138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7235d3e-869d-47ad-9fc9-2f7c131c8214 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:57.764415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7235d3e-869d-47ad-9fc9-2f7c131c8214 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:57.765094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7235d3e-869d-47ad-9fc9-2f7c131c8214 None admin] RBAC: Adding query filter params (name=service) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:57.765394 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7235d3e-869d-47ad-9fc9-2f7c131c8214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:57.766883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7235d3e-869d-47ad-9fc9-2f7c131c8214 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:57.780190 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 296/585] 10.222.0.36 () {56 vars in 1103 bytes} [Sun Oct 30 00:35:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:57.781187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7235d3e-869d-47ad-9fc9-2f7c131c8214 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:57.839784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:57.868645 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96773b7b-94e2-4694-b7b7-3454ba2f42a6 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:57.913272 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7235d3e-869d-47ad-9fc9-2f7c131c8214 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:57.913535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe7fe4-55ac-4555-baf8-2b6dc8263e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:57.914113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe7fe4-55ac-4555-baf8-2b6dc8263e96 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:57.914391 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe7fe4-55ac-4555-baf8-2b6dc8263e96 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:57.914661 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe7fe4-55ac-4555-baf8-2b6dc8263e96 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:57.915434 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe7fe4-55ac-4555-baf8-2b6dc8263e96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:57.917861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe7fe4-55ac-4555-baf8-2b6dc8263e96 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:57.932126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 290/586] 10.222.0.36 () {56 vars in 1087 bytes} [Sun Oct 30 00:35:57 2022] GET /identity/v3/services => generated 772 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:57.932126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe7fe4-55ac-4555-baf8-2b6dc8263e96 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:57.955733 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:57.970036 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeadba0-3532-411c-83c5-ac4af087a018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:57.970842 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeadba0-3532-411c-83c5-ac4af087a018 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:57.971127 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeadba0-3532-411c-83c5-ac4af087a018 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:57.971465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeadba0-3532-411c-83c5-ac4af087a018 None admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:57.972215 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeadba0-3532-411c-83c5-ac4af087a018 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:57.972571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeadba0-3532-411c-83c5-ac4af087a018 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:57.974093 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeadba0-3532-411c-83c5-ac4af087a018 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:57.991454 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 297/587] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:35:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:57.992445 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeadba0-3532-411c-83c5-ac4af087a018 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:58.083616 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe7fe4-55ac-4555-baf8-2b6dc8263e96 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:58.119036 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:58.124766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeadba0-3532-411c-83c5-ac4af087a018 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:58.132895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa4b26-52d7-414a-b195-bff1815362a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:58.133582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa4b26-52d7-414a-b195-bff1815362a9 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:58.133857 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa4b26-52d7-414a-b195-bff1815362a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:58.134131 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa4b26-52d7-414a-b195-bff1815362a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:58.148144 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa4b26-52d7-414a-b195-bff1815362a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:58.149616 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa4b26-52d7-414a-b195-bff1815362a9 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:58.165178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-8baa4b26-52d7-414a-b195-bff1815362a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:58.165178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:58.165178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:58.165178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:58.165178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:58.165178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:58.165178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:58.165178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:58.165178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:58.165178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:58.165178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:58.165178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:58.165178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:58.165178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:58.165178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:58.165178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:58.166530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:58.166530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:58.166530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:58.166530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:58.166530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:58.166530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:58.166530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:58.166530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:58.166530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:58.166530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:58.166530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:58.166530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:58.166530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:58.166530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:58.166530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:58.166530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:58.166530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:58.168145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:58.168145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:58.168145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:58.168145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 291/588] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:58.168145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa4b26-52d7-414a-b195-bff1815362a9 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:58.174135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f671a3-d97e-476c-b076-0d116631e6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:58.174816 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f671a3-d97e-476c-b076-0d116631e6bf None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:58.175101 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f671a3-d97e-476c-b076-0d116631e6bf None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:58.175374 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f671a3-d97e-476c-b076-0d116631e6bf None admin] PATH_INFO: `/v3/services/network` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:58.175962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f671a3-d97e-476c-b076-0d116631e6bf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:58.177399 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f671a3-d97e-476c-b076-0d116631e6bf None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:58.187904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-43f671a3-d97e-476c-b076-0d116631e6bf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 30 00:35:58.187904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:58.187904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:35:58.187904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:35:58.187904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:35:58.187904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:35:58.187904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:35:58.187904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:35:58.187904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 30 00:35:58.187904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:58.187904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:35:58.187904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:58.187904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:58.187904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:58.187904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:58.187904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:58.187904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:58.189228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:58.189228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:58.189228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:58.189228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:58.189228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:58.189228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:58.189228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:35:58.189228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:35:58.189228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:35:58.189228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:35:58.189228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:58.189228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:58.189228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:58.189228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:58.189228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:58.189228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:58.190109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:58.190109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:58.190109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:58.190109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:58.190109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:58.190109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:58.190109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:58.190109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:58.190109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:58.190109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:58.190109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:35:58.190109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:35:58.190109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 30 00:35:58.190109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:58.190109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 298/589] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:35:57 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:58.190917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374afb3a-829e-4290-bc28-8de7c661908e None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:58.190983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374afb3a-829e-4290-bc28-8de7c661908e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:58.191360 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374afb3a-829e-4290-bc28-8de7c661908e None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:58.192825 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 299/590] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:35:58.197534 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374afb3a-829e-4290-bc28-8de7c661908e None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:58.255805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:58.299079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa4b26-52d7-414a-b195-bff1815362a9 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:58.321738 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374afb3a-829e-4290-bc28-8de7c661908e None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:58.338562 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6644e5-e4be-49be-87e2-423dee1bcc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:58.339957 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6644e5-e4be-49be-87e2-423dee1bcc97 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:58.340331 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6644e5-e4be-49be-87e2-423dee1bcc97 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:58.340669 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6644e5-e4be-49be-87e2-423dee1bcc97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:58.341472 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6644e5-e4be-49be-87e2-423dee1bcc97 None admin] RBAC: Adding query filter params (role.id=980241d45fdb435d9f32c434057c71e4, scope.project.id=ab3a0e5fc5a344739b58ec8313639496, user.id=adbba7be39a64230bab160ce77ce7586) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:58.341780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6644e5-e4be-49be-87e2-423dee1bcc97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:58.343228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6644e5-e4be-49be-87e2-423dee1bcc97 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:58.352060 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 292/591] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:35:58 2022] GET /identity/v3/role_assignments?user.id=adbba7be39a64230bab160ce77ce7586&scope.project.id=ab3a0e5fc5a344739b58ec8313639496&role.id=980241d45fdb435d9f32c434057c71e4 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:58.354611 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6644e5-e4be-49be-87e2-423dee1bcc97 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:58.372656 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab8911-dd02-4122-b847-039e00c73ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:58.373569 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab8911-dd02-4122-b847-039e00c73ec0 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:58.374001 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab8911-dd02-4122-b847-039e00c73ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:58.374448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab8911-dd02-4122-b847-039e00c73ec0 None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:58.375287 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab8911-dd02-4122-b847-039e00c73ec0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:58.375754 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab8911-dd02-4122-b847-039e00c73ec0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:58.377631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab8911-dd02-4122-b847-039e00c73ec0 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:58.392966 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 300/592] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:58.394675 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ee95c-9af0-4580-9fa2-ec1ea5d908b1 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:58.395127 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ee95c-9af0-4580-9fa2-ec1ea5d908b1 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:58.395552 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ee95c-9af0-4580-9fa2-ec1ea5d908b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:58.429381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-645ee95c-9af0-4580-9fa2-ec1ea5d908b1 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:35:58.433157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-645ee95c-9af0-4580-9fa2-ec1ea5d908b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:35:58.454622 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:58.516426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6644e5-e4be-49be-87e2-423dee1bcc97 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:58.530095 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 301/593] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:35:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:35:58.531139 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ee95c-9af0-4580-9fa2-ec1ea5d908b1 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:58.568341 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:58.577350 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f2cde-2f1b-4ca2-8796-dccdf1cf672c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:58.578280 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f2cde-2f1b-4ca2-8796-dccdf1cf672c None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:58.578886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f2cde-2f1b-4ca2-8796-dccdf1cf672c None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:58.579347 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f2cde-2f1b-4ca2-8796-dccdf1cf672c None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:58.580440 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f2cde-2f1b-4ca2-8796-dccdf1cf672c None admin] RBAC: Adding query filter params (name=network) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:58.581004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f2cde-2f1b-4ca2-8796-dccdf1cf672c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:58.583618 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f2cde-2f1b-4ca2-8796-dccdf1cf672c None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:58.591211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 293/594] 10.222.0.36 () {56 vars in 1112 bytes} [Sun Oct 30 00:35:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:58.596828 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f2cde-2f1b-4ca2-8796-dccdf1cf672c None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:58.650401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ee95c-9af0-4580-9fa2-ec1ea5d908b1 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:58.691992 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9c337-3005-4f80-abe5-57179998e23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:58.693205 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9c337-3005-4f80-abe5-57179998e23e None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:58.693789 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9c337-3005-4f80-abe5-57179998e23e None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:58.694439 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9c337-3005-4f80-abe5-57179998e23e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:58.704713 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9c337-3005-4f80-abe5-57179998e23e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:58.706739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9c337-3005-4f80-abe5-57179998e23e None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:58.710411 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f2cde-2f1b-4ca2-8796-dccdf1cf672c None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:58.722735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-aeb9c337-3005-4f80-abe5-57179998e23e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 30 00:35:58.722735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:58.722735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:58.722735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:58.722735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:58.722735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:58.722735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:58.722735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:58.722735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:58.722735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:58.722735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:58.722735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:58.722735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:35:58.722735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:35:58.722735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:35:58.722735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:35:58.723749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:58.723749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:58.723749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:35:58.723749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:58.723749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:35:58.723749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:35:58.723749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:58.723749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:58.723749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:58.723749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:58.723749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:58.723749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:58.723749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:58.723749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:58.723749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:58.723749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:58.723749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:58.723749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:58.724763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:58.724763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:58.724763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:35:58.724763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:35:58.724763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:35:58.724763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:35:58.724763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:35:58.724763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:35:58.724763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 30 00:35:58.724763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:58.727674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 302/595] 10.222.0.36 () {56 vars in 1117 bytes} [Sun Oct 30 00:35:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:58.734938 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9c337-3005-4f80-abe5-57179998e23e None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:58.749420 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dc961-c0fe-4493-a588-f0606c2b5436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:58.750329 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dc961-c0fe-4493-a588-f0606c2b5436 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:58.750329 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dc961-c0fe-4493-a588-f0606c2b5436 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:58.750600 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dc961-c0fe-4493-a588-f0606c2b5436 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:58.751215 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645dc961-c0fe-4493-a588-f0606c2b5436 None admin] RBAC: Adding query filter params (type=network) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:58.751517 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645dc961-c0fe-4493-a588-f0606c2b5436 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:58.752953 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645dc961-c0fe-4493-a588-f0606c2b5436 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:58.757788 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:58.758572 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 294/596] 10.222.0.36 () {56 vars in 1112 bytes} [Sun Oct 30 00:35:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:35:58.765167 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dc961-c0fe-4493-a588-f0606c2b5436 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:58.832238 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:58.876264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9c337-3005-4f80-abe5-57179998e23e None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:58.879913 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dc961-c0fe-4493-a588-f0606c2b5436 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:58.918273 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa36fb9-1ca2-48a7-9eb7-ddd9b6ede76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:58.918825 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa36fb9-1ca2-48a7-9eb7-ddd9b6ede76f None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:58.919104 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa36fb9-1ca2-48a7-9eb7-ddd9b6ede76f None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:58.919376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa36fb9-1ca2-48a7-9eb7-ddd9b6ede76f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:58.920107 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa36fb9-1ca2-48a7-9eb7-ddd9b6ede76f None admin] RBAC: Adding query filter params (interface=public, service_id=d3a002dfb58f45fca988b3e049a9652d, region_id=RegionOne) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:58.920427 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa36fb9-1ca2-48a7-9eb7-ddd9b6ede76f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:58.922000 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa36fb9-1ca2-48a7-9eb7-ddd9b6ede76f None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:58.928480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 295/597] 10.222.0.36 () {56 vars in 1251 bytes} [Sun Oct 30 00:35:58 2022] GET /identity/v3/endpoints?service_id=d3a002dfb58f45fca988b3e049a9652d&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:58.929294 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa36fb9-1ca2-48a7-9eb7-ddd9b6ede76f None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:58.939190 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a4d3c8-1cb9-461e-8e3b-398f89ca9604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:58.939832 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4d3c8-1cb9-461e-8e3b-398f89ca9604 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:58.940133 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4d3c8-1cb9-461e-8e3b-398f89ca9604 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:58.940415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4d3c8-1cb9-461e-8e3b-398f89ca9604 None admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:58.941168 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a4d3c8-1cb9-461e-8e3b-398f89ca9604 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:58.941473 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a4d3c8-1cb9-461e-8e3b-398f89ca9604 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:58.943032 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a4d3c8-1cb9-461e-8e3b-398f89ca9604 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:58.966068 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 303/598] 10.222.0.36 () {56 vars in 1149 bytes} [Sun Oct 30 00:35:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:58.967414 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a4d3c8-1cb9-461e-8e3b-398f89ca9604 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:59.071471 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:59.074238 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa36fb9-1ca2-48a7-9eb7-ddd9b6ede76f None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:59.102334 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:59.116453 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a4d3c8-1cb9-461e-8e3b-398f89ca9604 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:59.116681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412cf36-e82c-458e-9480-c5fa06afcf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:59.117123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412cf36-e82c-458e-9480-c5fa06afcf4e None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:59.117400 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412cf36-e82c-458e-9480-c5fa06afcf4e None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:59.117668 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412cf36-e82c-458e-9480-c5fa06afcf4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:59.130172 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2412cf36-e82c-458e-9480-c5fa06afcf4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:35:59.136678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-2412cf36-e82c-458e-9480-c5fa06afcf4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:35:59.136678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:59.136678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:59.136678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:59.136678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:59.136678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:59.136678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:59.136678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:59.136678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:59.136678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:59.136678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:59.136678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:59.136678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:35:59.136678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:35:59.136678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:35:59.136678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:35:59.137765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:35:59.137765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:35:59.137765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:35:59.137765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:35:59.137765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:59.137765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:59.137765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:59.137765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:59.137765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:59.137765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:59.137765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:59.137765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:59.137765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:59.137765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:59.137765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:59.137765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:59.138898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:59.138898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:59.138898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:59.138898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:59.138898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:35:59.138898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:35:59.138898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:35:59.138898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:35:59.138898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:35:59.138898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:35:59.138898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:35:59.138898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:35:59.138898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 296/599] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:35:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:59.139821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412cf36-e82c-458e-9480-c5fa06afcf4e None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:59.164880 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51233c-810d-4cf3-97c7-6fcfb81eb2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:59.166599 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51233c-810d-4cf3-97c7-6fcfb81eb2e2 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:59.167092 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51233c-810d-4cf3-97c7-6fcfb81eb2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:59.167515 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51233c-810d-4cf3-97c7-6fcfb81eb2e2 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:59.168655 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51233c-810d-4cf3-97c7-6fcfb81eb2e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:59.170770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51233c-810d-4cf3-97c7-6fcfb81eb2e2 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:59.178363 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 304/600] 10.222.0.36 () {56 vars in 1087 bytes} [Sun Oct 30 00:35:58 2022] GET /identity/v3/services => generated 772 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:59.179579 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51233c-810d-4cf3-97c7-6fcfb81eb2e2 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:59.304876 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412cf36-e82c-458e-9480-c5fa06afcf4e None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:59.365277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff55ba-bfce-4f40-a875-a48c31232f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:59.366249 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff55ba-bfce-4f40-a875-a48c31232f84 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:59.366537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff55ba-bfce-4f40-a875-a48c31232f84 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:59.366813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff55ba-bfce-4f40-a875-a48c31232f84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:59.444045 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:59.444918 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51233c-810d-4cf3-97c7-6fcfb81eb2e2 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:59.473851 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff55ba-bfce-4f40-a875-a48c31232f84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:59.475382 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff55ba-bfce-4f40-a875-a48c31232f84 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:59.529669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4f16e-113e-4608-96c3-956fefca9959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:59.530302 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4f16e-113e-4608-96c3-956fefca9959 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:59.530579 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4f16e-113e-4608-96c3-956fefca9959 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:59.530850 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4f16e-113e-4608-96c3-956fefca9959 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:59.531524 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4f16e-113e-4608-96c3-956fefca9959 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:35:59.531825 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4f16e-113e-4608-96c3-956fefca9959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:59.533248 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4f16e-113e-4608-96c3-956fefca9959 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:59.537027 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:59.553761 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 305/601] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:35:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:35:59.559435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4f16e-113e-4608-96c3-956fefca9959 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:59.689559 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4f16e-113e-4608-96c3-956fefca9959 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:35:59.736133 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9524f-ada3-48ec-9adc-e8fa766fd1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:35:59.736769 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9524f-ada3-48ec-9adc-e8fa766fd1db None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:35:59.737052 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9524f-ada3-48ec-9adc-e8fa766fd1db None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:35:59.737325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9524f-ada3-48ec-9adc-e8fa766fd1db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:35:59.744392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 297/602] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:35:59 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 30 00:35:59.750517 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff55ba-bfce-4f40-a875-a48c31232f84 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:59.751720 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9524f-ada3-48ec-9adc-e8fa766fd1db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:35:59.753641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9524f-ada3-48ec-9adc-e8fa766fd1db None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:35:59.773660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-c7d9524f-ada3-48ec-9adc-e8fa766fd1db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:59.773660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:35:59.773660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:35:59.773660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:35:59.773660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:35:59.773660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:35:59.773660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:35:59.773660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:35:59.773660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:35:59.773660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:35:59.773660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:35:59.773660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:35:59.773660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:35:59.773660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:35:59.773660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:35:59.773660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:35:59.774667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:35:59.774667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:35:59.774667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:35:59.774667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:35:59.774667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:35:59.774667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:35:59.774667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:35:59.774667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:35:59.774667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:35:59.774667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:35:59.774667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:35:59.774667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:35:59.774667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:35:59.774667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:35:59.774667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:35:59.774667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:35:59.774667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:35:59.776048 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:35:59.776048 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:35:59.776048 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:35:59.776048 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 306/603] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:35:59.783531 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9524f-ada3-48ec-9adc-e8fa766fd1db None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:35:59.904773 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:59.947339 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:35:59.977274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff55ba-bfce-4f40-a875-a48c31232f84 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:00.001546 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9524f-ada3-48ec-9adc-e8fa766fd1db None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:00.049435 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea39ed-6c0e-4acc-a676-9928428067c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:00.050150 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea39ed-6c0e-4acc-a676-9928428067c8 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:00.050433 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea39ed-6c0e-4acc-a676-9928428067c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:00.050708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea39ed-6c0e-4acc-a676-9928428067c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:00.067761 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fdf25-b3d5-4b18-9dcc-fa24e87710cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:00.068412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fdf25-b3d5-4b18-9dcc-fa24e87710cd None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:00.068694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fdf25-b3d5-4b18-9dcc-fa24e87710cd None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:00.068969 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fdf25-b3d5-4b18-9dcc-fa24e87710cd None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:00.069624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fdf25-b3d5-4b18-9dcc-fa24e87710cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:00.069935 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fdf25-b3d5-4b18-9dcc-fa24e87710cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:00.071340 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fdf25-b3d5-4b18-9dcc-fa24e87710cd None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:00.074766 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea39ed-6c0e-4acc-a676-9928428067c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:00.076276 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea39ed-6c0e-4acc-a676-9928428067c8 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:00.105496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-86ea39ed-6c0e-4acc-a676-9928428067c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:00.105496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:00.105496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:00.105496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:00.105496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:00.105496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:00.105496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:00.105496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:00.105496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:00.105496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:00.105496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:00.105496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:00.105496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:36:00.105496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:36:00.105496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:36:00.105496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:36:00.106628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:00.106628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:00.106628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:00.106628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:00.106628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:00.106628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:00.106628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:00.106628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:00.106628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:00.106628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:00.106628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:00.106628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:00.106628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:00.106628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:00.106628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:00.106628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:00.106628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:36:00.108017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:36:00.108017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:00.108017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:00.108017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 298/604] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:36:00.108341 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 307/605] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:00.109382 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea39ed-6c0e-4acc-a676-9928428067c8 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:00.112716 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fdf25-b3d5-4b18-9dcc-fa24e87710cd None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:00.266287 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fdf25-b3d5-4b18-9dcc-fa24e87710cd None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:00.288464 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:00.333780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea39ed-6c0e-4acc-a676-9928428067c8 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:00.346860 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d60f3-c475-4cfc-9f4a-4f751fc5fe82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:00.347495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d60f3-c475-4cfc-9f4a-4f751fc5fe82 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:00.347772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d60f3-c475-4cfc-9f4a-4f751fc5fe82 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:00.348041 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d60f3-c475-4cfc-9f4a-4f751fc5fe82 None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:00.348723 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d60f3-c475-4cfc-9f4a-4f751fc5fe82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:00.349026 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d60f3-c475-4cfc-9f4a-4f751fc5fe82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:00.350430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d60f3-c475-4cfc-9f4a-4f751fc5fe82 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:00.379833 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 308/606] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:00.380863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d60f3-c475-4cfc-9f4a-4f751fc5fe82 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:00.409912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b89b24-4a0f-48e8-a41d-08fb06feca48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:00.410560 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b89b24-4a0f-48e8-a41d-08fb06feca48 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:00.410834 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b89b24-4a0f-48e8-a41d-08fb06feca48 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:00.411106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b89b24-4a0f-48e8-a41d-08fb06feca48 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:00.416590 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:00.426856 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b89b24-4a0f-48e8-a41d-08fb06feca48 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:00.428410 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b89b24-4a0f-48e8-a41d-08fb06feca48 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:00.441181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-70b89b24-4a0f-48e8-a41d-08fb06feca48 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 30 00:36:00.441181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:00.441181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:00.441181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:00.441181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:00.441181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:00.441181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:00.441181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:00.441181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:00.441181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:00.441181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:00.441181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:00.441181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:36:00.441181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:36:00.441181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:36:00.441181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:36:00.442409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:00.442409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:00.442409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:36:00.442409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:36:00.442409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:36:00.442409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:36:00.442409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:00.442409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:00.442409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:00.442409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:00.442409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:00.442409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:00.442409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:00.442409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:00.442409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:00.442409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:00.442409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:00.442409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:00.443767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:00.443767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:00.443767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:36:00.443767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:36:00.443767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:36:00.443767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:36:00.443767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:36:00.443767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:36:00.443767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 30 00:36:00.443767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:00.443767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 299/607] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:36:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:00.444508 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea8452-330e-411d-a63d-b2997e54e342 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:00.444508 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea8452-330e-411d-a63d-b2997e54e342 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:00.444508 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea8452-330e-411d-a63d-b2997e54e342 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:00.450006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 300/608] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:00.450006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea8452-330e-411d-a63d-b2997e54e342 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:00.540192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d60f3-c475-4cfc-9f4a-4f751fc5fe82 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:00.588740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea8452-330e-411d-a63d-b2997e54e342 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:00.590816 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24f61b-b58e-4981-80f3-ae915fd4e5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:00.592942 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24f61b-b58e-4981-80f3-ae915fd4e5db None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:00.592942 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24f61b-b58e-4981-80f3-ae915fd4e5db None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:00.593164 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24f61b-b58e-4981-80f3-ae915fd4e5db None admin] PATH_INFO: `/v3/projects/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:00.601965 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24f61b-b58e-4981-80f3-ae915fd4e5db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:00.603504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24f61b-b58e-4981-80f3-ae915fd4e5db None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:00.619894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-fd24f61b-b58e-4981-80f3-ae915fd4e5db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:36:00.619894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:00.619894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:00.619894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:00.619894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:00.619894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:00.619894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:00.619894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:00.619894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:00.619894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:00.619894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:00.619894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:00.619894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:36:00.619894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:36:00.619894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:36:00.621415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:36:00.621415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:00.621415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:00.621415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:00.621415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:00.621415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:00.621415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:00.621415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:00.621415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:00.621415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:00.621415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:00.621415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:00.621415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:00.621415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:00.621415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:00.621415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:00.621415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:00.622304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:36:00.622304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:36:00.622304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:36:00.622304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:36:00.622304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:36:00.622304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:36:00.622304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:36:00.622304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:00.623414 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 309/609] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:36:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:00.623766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615e304-2240-4333-945d-69f71721fe4e None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:00.624049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615e304-2240-4333-945d-69f71721fe4e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:00.624342 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615e304-2240-4333-945d-69f71721fe4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:00.632545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1dee5e-edcb-4a2e-9777-faec8af241dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:00.633507 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1dee5e-edcb-4a2e-9777-faec8af241dc None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:00.634004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1dee5e-edcb-4a2e-9777-faec8af241dc None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:00.634476 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1dee5e-edcb-4a2e-9777-faec8af241dc None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:00.635419 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1dee5e-edcb-4a2e-9777-faec8af241dc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:00.636499 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1dee5e-edcb-4a2e-9777-faec8af241dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:00.638170 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1dee5e-edcb-4a2e-9777-faec8af241dc None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:00.649466 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-5615e304-2240-4333-945d-69f71721fe4e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:00.650141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-5615e304-2240-4333-945d-69f71721fe4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:00.658566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 301/610] 10.222.0.36 () {56 vars in 1135 bytes} [Sun Oct 30 00:36:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:00.659806 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1dee5e-edcb-4a2e-9777-faec8af241dc None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:00.684764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:00.724793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:00.752487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 310/611] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:00.752487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615e304-2240-4333-945d-69f71721fe4e None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:00.787533 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1dee5e-edcb-4a2e-9777-faec8af241dc None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:00.863669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615e304-2240-4333-945d-69f71721fe4e None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:00.892825 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41677d-2676-4dd6-98fb-8c7237588798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:00.893860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41677d-2676-4dd6-98fb-8c7237588798 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:00.894346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41677d-2676-4dd6-98fb-8c7237588798 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:00.894822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41677d-2676-4dd6-98fb-8c7237588798 None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:00.895731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41677d-2676-4dd6-98fb-8c7237588798 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:00.896274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41677d-2676-4dd6-98fb-8c7237588798 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:00.897946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41677d-2676-4dd6-98fb-8c7237588798 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:00.913611 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13f8fe-cb67-4ad9-b2b6-179fe24055ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:00.914236 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13f8fe-cb67-4ad9-b2b6-179fe24055ee None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:00.914511 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13f8fe-cb67-4ad9-b2b6-179fe24055ee None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:00.914883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13f8fe-cb67-4ad9-b2b6-179fe24055ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:00.931337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 302/612] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:36:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:00.932346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41677d-2676-4dd6-98fb-8c7237588798 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:00.997660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:01.086230 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13f8fe-cb67-4ad9-b2b6-179fe24055ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:01.089660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13f8fe-cb67-4ad9-b2b6-179fe24055ee None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:01.131107 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41677d-2676-4dd6-98fb-8c7237588798 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:01.149779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:01.175030 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25faaaa4-4c9b-44f3-9c30-064cf9bd5edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:01.175646 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25faaaa4-4c9b-44f3-9c30-064cf9bd5edb None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:01.175924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25faaaa4-4c9b-44f3-9c30-064cf9bd5edb None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:01.176216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25faaaa4-4c9b-44f3-9c30-064cf9bd5edb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:01.176954 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25faaaa4-4c9b-44f3-9c30-064cf9bd5edb None admin] RBAC: Adding query filter params (role.id=9a8de43206fa4a8fa4901488bf86f9f5, scope.project.id=ab3a0e5fc5a344739b58ec8313639496, user.id=8a91d3b9c4834d729211ec367e235889) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:01.177253 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25faaaa4-4c9b-44f3-9c30-064cf9bd5edb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:01.178755 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25faaaa4-4c9b-44f3-9c30-064cf9bd5edb None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:01.186508 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 303/613] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:36:00 2022] GET /identity/v3/role_assignments?user.id=8a91d3b9c4834d729211ec367e235889&scope.project.id=ab3a0e5fc5a344739b58ec8313639496&role.id=9a8de43206fa4a8fa4901488bf86f9f5 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:01.187374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25faaaa4-4c9b-44f3-9c30-064cf9bd5edb None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:01.257173 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 311/614] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:36:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 30 00:36:01.264591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13f8fe-cb67-4ad9-b2b6-179fe24055ee None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:01.314369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25faaaa4-4c9b-44f3-9c30-064cf9bd5edb None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:01.361288 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508114f2-5419-41bd-8f6f-f32199b2eb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:01.361945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508114f2-5419-41bd-8f6f-f32199b2eb02 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:01.362211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508114f2-5419-41bd-8f6f-f32199b2eb02 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:01.362484 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508114f2-5419-41bd-8f6f-f32199b2eb02 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:01.363210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508114f2-5419-41bd-8f6f-f32199b2eb02 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:01.364862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508114f2-5419-41bd-8f6f-f32199b2eb02 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:01.375841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-508114f2-5419-41bd-8f6f-f32199b2eb02 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 30 00:36:01.375841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:01.375841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:01.375841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:01.375841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:01.375841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:01.375841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:01.375841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:01.375841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 30 00:36:01.375841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:01.375841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:01.375841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:01.375841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:01.375841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:01.375841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:01.375841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:01.375841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:01.376959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:01.376959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:01.376959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:01.376959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:01.376959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:01.376959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:01.376959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:01.376959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:01.376959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:01.376959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:01.376959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:01.376959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:01.376959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:01.376959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:01.376959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:01.376959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:01.378022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:01.378022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:01.378022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:01.378022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:01.378022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:01.378022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:01.378022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:01.378022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:01.378022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:01.378022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:01.378022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:01.378022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:01.378022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 30 00:36:01.378022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:01.378022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 304/615] 10.222.0.36 () {56 vars in 1129 bytes} [Sun Oct 30 00:36:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:01.378821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337893d9-b6c6-4bb3-b795-c97e96632a9b None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:01.379740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337893d9-b6c6-4bb3-b795-c97e96632a9b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:01.379740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337893d9-b6c6-4bb3-b795-c97e96632a9b None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:01.386421 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 305/616] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:01.387551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337893d9-b6c6-4bb3-b795-c97e96632a9b None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:01.430336 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:01.441223 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:01.455740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13f8fe-cb67-4ad9-b2b6-179fe24055ee None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:01.502413 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d511f81-9842-4a84-ab57-714ecee16669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:01.503114 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d511f81-9842-4a84-ab57-714ecee16669 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:01.503444 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d511f81-9842-4a84-ab57-714ecee16669 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:01.503721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d511f81-9842-4a84-ab57-714ecee16669 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:01.515920 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d511f81-9842-4a84-ab57-714ecee16669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:01.517472 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d511f81-9842-4a84-ab57-714ecee16669 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:01.517587 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337893d9-b6c6-4bb3-b795-c97e96632a9b None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:01.533392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-8d511f81-9842-4a84-ab57-714ecee16669 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:01.533392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:01.533392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:01.533392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:01.533392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:01.533392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:01.533392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:01.533392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:01.533392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:01.533392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:01.533392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:01.533392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:01.533392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:36:01.533392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:36:01.533392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:36:01.533392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:36:01.534434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:01.534434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:01.534434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:01.534434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:01.534434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:01.534434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:01.534434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:01.534434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:01.534434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:01.534434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:01.534434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:01.534434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:01.534434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:01.534434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:01.534434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:01.534434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:01.534434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:36:01.535329 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:36:01.535329 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:01.535329 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:01.535329 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 312/617] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:01.536232 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da0295-27e3-4edb-915b-0f5e59b5438e None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:01.536520 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da0295-27e3-4edb-915b-0f5e59b5438e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:01.536793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da0295-27e3-4edb-915b-0f5e59b5438e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:01.557264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-80da0295-27e3-4edb-915b-0f5e59b5438e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:01.558541 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-80da0295-27e3-4edb-915b-0f5e59b5438e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:01.560897 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7899f53-34f1-4fcd-a0dc-cb6981f6fc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:01.561943 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7899f53-34f1-4fcd-a0dc-cb6981f6fc22 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:01.562225 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7899f53-34f1-4fcd-a0dc-cb6981f6fc22 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:01.562636 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7899f53-34f1-4fcd-a0dc-cb6981f6fc22 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:01.563337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7899f53-34f1-4fcd-a0dc-cb6981f6fc22 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:01.563737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7899f53-34f1-4fcd-a0dc-cb6981f6fc22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:01.565286 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7899f53-34f1-4fcd-a0dc-cb6981f6fc22 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:01.570901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 306/618] 10.222.0.36 () {56 vars in 1124 bytes} [Sun Oct 30 00:36:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:01.577103 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7899f53-34f1-4fcd-a0dc-cb6981f6fc22 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:01.651221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 313/619] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:01.653035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da0295-27e3-4edb-915b-0f5e59b5438e None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:01.742229 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:01.756146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7899f53-34f1-4fcd-a0dc-cb6981f6fc22 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:01.763644 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:01.801441 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125a768-4a38-4fd2-8654-43ea4ca7f559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:01.802061 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125a768-4a38-4fd2-8654-43ea4ca7f559 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:01.802344 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125a768-4a38-4fd2-8654-43ea4ca7f559 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:01.802654 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125a768-4a38-4fd2-8654-43ea4ca7f559 None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:01.803263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4125a768-4a38-4fd2-8654-43ea4ca7f559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:01.803567 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4125a768-4a38-4fd2-8654-43ea4ca7f559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:01.806121 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4125a768-4a38-4fd2-8654-43ea4ca7f559 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:01.827972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 307/620] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:01.827972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd5bec-cdba-4933-a0fc-afea5b6c2409 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:01.827972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd5bec-cdba-4933-a0fc-afea5b6c2409 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:01.827972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd5bec-cdba-4933-a0fc-afea5b6c2409 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:01.827972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 308/621] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:01.832913 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f836ea-aa7a-4581-b789-66680ef305f2 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:01.833250 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f836ea-aa7a-4581-b789-66680ef305f2 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:01.833528 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f836ea-aa7a-4581-b789-66680ef305f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:01.858011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-19f836ea-aa7a-4581-b789-66680ef305f2 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:01.859226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-19f836ea-aa7a-4581-b789-66680ef305f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:01.863791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da0295-27e3-4edb-915b-0f5e59b5438e None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:01.919631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108cce7-d2eb-4a5d-8bea-f5557f8cc36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:01.920650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108cce7-d2eb-4a5d-8bea-f5557f8cc36b None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:01.921138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108cce7-d2eb-4a5d-8bea-f5557f8cc36b None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:01.921630 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108cce7-d2eb-4a5d-8bea-f5557f8cc36b None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:01.922695 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108cce7-d2eb-4a5d-8bea-f5557f8cc36b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:01.923496 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108cce7-d2eb-4a5d-8bea-f5557f8cc36b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:01.925980 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108cce7-d2eb-4a5d-8bea-f5557f8cc36b None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:01.933646 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 314/622] 10.222.0.36 () {56 vars in 1124 bytes} [Sun Oct 30 00:36:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:01.935274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108cce7-d2eb-4a5d-8bea-f5557f8cc36b None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:01.952139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 309/623] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:01.952139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f836ea-aa7a-4581-b789-66680ef305f2 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:02.089556 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:02.093727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108cce7-d2eb-4a5d-8bea-f5557f8cc36b None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:02.099822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f836ea-aa7a-4581-b789-66680ef305f2 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:02.108349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:02.145208 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6818091-edca-4cd7-ac35-c44d9cbac95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:02.146622 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6818091-edca-4cd7-ac35-c44d9cbac95c None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:02.146914 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6818091-edca-4cd7-ac35-c44d9cbac95c None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:02.147270 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6818091-edca-4cd7-ac35-c44d9cbac95c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:02.147665 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40faac8-a092-41bf-95fc-9da380945023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:02.148599 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40faac8-a092-41bf-95fc-9da380945023 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:02.148996 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40faac8-a092-41bf-95fc-9da380945023 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:02.149562 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40faac8-a092-41bf-95fc-9da380945023 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:02.150694 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40faac8-a092-41bf-95fc-9da380945023 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:02.152787 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40faac8-a092-41bf-95fc-9da380945023 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:02.157678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6818091-edca-4cd7-ac35-c44d9cbac95c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:02.159128 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6818091-edca-4cd7-ac35-c44d9cbac95c None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:02.173157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-a6818091-edca-4cd7-ac35-c44d9cbac95c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:36:02.173157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:02.173157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:02.173157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:02.173157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:02.173157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:02.173157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:02.173157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:02.173157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:02.173157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:02.173157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:02.173157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:02.173157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:36:02.173157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:36:02.173157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:36:02.174346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:36:02.174346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:02.174346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:02.174346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:02.174346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:02.174346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:02.174346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:02.174346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:02.174346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:02.174346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:02.174346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:02.174346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:02.174346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:02.174346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:02.174346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:02.174346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:02.174346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:02.175561 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:36:02.175561 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:36:02.175561 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:36:02.175561 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:36:02.175561 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:36:02.175561 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:36:02.175561 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:36:02.175561 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:02.183251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 315/624] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:36:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:02.184415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6818091-edca-4cd7-ac35-c44d9cbac95c None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:02.192586 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 310/625] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:36:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:36:02.194380 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40faac8-a092-41bf-95fc-9da380945023 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:02.313211 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6818091-edca-4cd7-ac35-c44d9cbac95c None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:02.361450 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fdb32-0b34-4937-b7ae-7ae9e9711ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:02.362793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fdb32-0b34-4937-b7ae-7ae9e9711ed4 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:02.363282 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fdb32-0b34-4937-b7ae-7ae9e9711ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:02.363872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fdb32-0b34-4937-b7ae-7ae9e9711ed4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:02.364604 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40faac8-a092-41bf-95fc-9da380945023 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:02.365317 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64fdb32-0b34-4937-b7ae-7ae9e9711ed4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:02.369432 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64fdb32-0b34-4937-b7ae-7ae9e9711ed4 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:02.382349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-f64fdb32-0b34-4937-b7ae-7ae9e9711ed4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 30 00:36:02.382349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:02.382349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:02.382349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:02.382349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:02.382349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:02.382349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:02.382349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:02.382349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 30 00:36:02.382349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:02.382349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:02.382349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:02.382349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:02.382349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:02.382349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:02.382349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:02.383553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:02.383553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:02.383553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:02.383553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:02.383553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:02.383553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:02.383553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:02.383553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:02.383553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:02.383553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:02.383553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:02.383553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:02.383553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:02.383553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:02.383553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:02.383553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:02.383553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:02.384679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:02.384679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:02.384679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:02.384679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:02.384679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:02.384679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:02.384679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:02.384679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:02.384679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:02.384679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:02.384679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:02.384679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:02.384679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 30 00:36:02.384679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:02.386591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 316/626] 10.222.0.36 () {56 vars in 1132 bytes} [Sun Oct 30 00:36:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:02.388485 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fdb32-0b34-4937-b7ae-7ae9e9711ed4 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:02.414630 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd59a25-7400-4752-9368-d0a82dbb93a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:02.415655 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd59a25-7400-4752-9368-d0a82dbb93a7 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:02.416011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd59a25-7400-4752-9368-d0a82dbb93a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:02.416637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd59a25-7400-4752-9368-d0a82dbb93a7 None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:02.418056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd59a25-7400-4752-9368-d0a82dbb93a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:02.418456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd59a25-7400-4752-9368-d0a82dbb93a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:02.420188 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd59a25-7400-4752-9368-d0a82dbb93a7 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:02.428309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:02.437667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 311/627] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:36:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:02.437667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd59a25-7400-4752-9368-d0a82dbb93a7 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:02.452036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:02.533770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fdb32-0b34-4937-b7ae-7ae9e9711ed4 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:02.574506 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9c863-69f6-4110-b0fc-c43b185a58c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:02.585384 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9c863-69f6-4110-b0fc-c43b185a58c7 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:02.585384 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9c863-69f6-4110-b0fc-c43b185a58c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:02.585603 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd59a25-7400-4752-9368-d0a82dbb93a7 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:02.585709 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9c863-69f6-4110-b0fc-c43b185a58c7 None admin] PATH_INFO: `/v3/services/network` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:02.585709 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9c863-69f6-4110-b0fc-c43b185a58c7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:02.585709 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9c863-69f6-4110-b0fc-c43b185a58c7 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:02.593085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-5de9c863-69f6-4110-b0fc-c43b185a58c7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 30 00:36:02.593085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:02.593085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:02.593085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:02.593085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:02.593085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:02.593085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:02.593085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:02.593085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 30 00:36:02.593085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:02.593085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:02.593085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:02.593085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:02.593085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:02.593085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:02.593085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:02.593085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:02.594217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:02.594217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:02.594217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:02.594217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:02.594217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:02.594217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:02.594217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:02.594217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:02.594217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:02.594217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:02.594217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:02.594217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:02.594217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:02.594217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:02.594217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:02.594217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:02.595135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:02.595135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:02.595135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:02.595135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:02.595135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:02.595135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:02.595135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:02.595135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:02.595135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:02.595135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:02.595135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:02.595135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:02.595135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 30 00:36:02.595135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:02.595135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 317/628] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:36:02 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:02.597108 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9c863-69f6-4110-b0fc-c43b185a58c7 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:02.631129 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483cab76-a4a3-40e9-b218-cb963284b487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:02.632060 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cab76-a4a3-40e9-b218-cb963284b487 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:02.632578 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cab76-a4a3-40e9-b218-cb963284b487 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:02.633048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cab76-a4a3-40e9-b218-cb963284b487 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:02.634179 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483cab76-a4a3-40e9-b218-cb963284b487 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:02.634690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483cab76-a4a3-40e9-b218-cb963284b487 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:02.636826 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483cab76-a4a3-40e9-b218-cb963284b487 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:02.645227 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 312/629] 10.222.0.36 () {56 vars in 1126 bytes} [Sun Oct 30 00:36:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:02.646141 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483cab76-a4a3-40e9-b218-cb963284b487 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:02.716531 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:02.732407 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9c863-69f6-4110-b0fc-c43b185a58c7 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:02.743464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:02.772845 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9eb4be-853b-4679-a3d8-d0e9d59ee341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:02.773867 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9eb4be-853b-4679-a3d8-d0e9d59ee341 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:02.774354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9eb4be-853b-4679-a3d8-d0e9d59ee341 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:02.774820 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9eb4be-853b-4679-a3d8-d0e9d59ee341 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:02.775812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9eb4be-853b-4679-a3d8-d0e9d59ee341 None admin] RBAC: Adding query filter params (role.id=980241d45fdb435d9f32c434057c71e4, scope.project.id=ab3a0e5fc5a344739b58ec8313639496, user.id=9f4c779b8422432fb06cdf996841b9b5) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:02.776372 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9eb4be-853b-4679-a3d8-d0e9d59ee341 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:02.778482 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9eb4be-853b-4679-a3d8-d0e9d59ee341 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:02.785059 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483cab76-a4a3-40e9-b218-cb963284b487 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:02.785621 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 318/630] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:36:02 2022] GET /identity/v3/role_assignments?user.id=9f4c779b8422432fb06cdf996841b9b5&scope.project.id=ab3a0e5fc5a344739b58ec8313639496&role.id=980241d45fdb435d9f32c434057c71e4 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:02.786847 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9eb4be-853b-4679-a3d8-d0e9d59ee341 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:02.840523 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d4a82-4800-4d61-ad3e-a793b452339f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:02.843927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d4a82-4800-4d61-ad3e-a793b452339f None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:02.846117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d4a82-4800-4d61-ad3e-a793b452339f None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:02.847350 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d4a82-4800-4d61-ad3e-a793b452339f None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:02.850176 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d4a82-4800-4d61-ad3e-a793b452339f None admin] RBAC: Adding query filter params (name=network) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:02.851415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d4a82-4800-4d61-ad3e-a793b452339f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:02.852867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d4a82-4800-4d61-ad3e-a793b452339f None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:02.864124 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 313/631] 10.222.0.36 () {56 vars in 1112 bytes} [Sun Oct 30 00:36:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:02.866604 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d4a82-4800-4d61-ad3e-a793b452339f None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:02.993235 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9eb4be-853b-4679-a3d8-d0e9d59ee341 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:03.021429 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d4a82-4800-4d61-ad3e-a793b452339f None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:03.069765 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888ed7d-e970-4150-b597-e8c54fb12606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:03.070404 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888ed7d-e970-4150-b597-e8c54fb12606 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:03.070682 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888ed7d-e970-4150-b597-e8c54fb12606 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:03.070968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888ed7d-e970-4150-b597-e8c54fb12606 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:03.071605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888ed7d-e970-4150-b597-e8c54fb12606 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:03.071908 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888ed7d-e970-4150-b597-e8c54fb12606 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:03.073291 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102bb333-855c-466a-9ef8-50885538c062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:03.074691 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bb333-855c-466a-9ef8-50885538c062 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:03.075075 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bb333-855c-466a-9ef8-50885538c062 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:03.075349 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bb333-855c-466a-9ef8-50885538c062 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:03.076005 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102bb333-855c-466a-9ef8-50885538c062 None admin] RBAC: Adding query filter params (type=network) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:03.076329 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102bb333-855c-466a-9ef8-50885538c062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:03.077725 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102bb333-855c-466a-9ef8-50885538c062 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:03.079660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888ed7d-e970-4150-b597-e8c54fb12606 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:03.084523 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:03.086769 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 319/632] 10.222.0.36 () {56 vars in 1126 bytes} [Sun Oct 30 00:36:02 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:03.094872 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 314/633] 10.222.0.36 () {56 vars in 1112 bytes} [Sun Oct 30 00:36:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:03.100720 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888ed7d-e970-4150-b597-e8c54fb12606 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:03.125024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:03.228995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888ed7d-e970-4150-b597-e8c54fb12606 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:03.274392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e6d01-8e04-4293-be5c-8bfe6c8e574b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:03.275010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e6d01-8e04-4293-be5c-8bfe6c8e574b None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:03.275285 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e6d01-8e04-4293-be5c-8bfe6c8e574b None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:03.275551 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e6d01-8e04-4293-be5c-8bfe6c8e574b None admin] PATH_INFO: `/v3/endpoints` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:03.276263 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e6d01-8e04-4293-be5c-8bfe6c8e574b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:03.278204 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e6d01-8e04-4293-be5c-8bfe6c8e574b None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:03.308667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 320/634] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:36:03 2022] POST /identity/v3/endpoints => generated 337 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:36:03.659664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50bf4d-ee2a-4dc2-b843-a14501ba6033 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:03.660231 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50bf4d-ee2a-4dc2-b843-a14501ba6033 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:03.660231 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50bf4d-ee2a-4dc2-b843-a14501ba6033 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:03.661707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 315/635] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:03.668433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c30f5-23c5-462d-8e3a-a68013dd28d6 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:03.669311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c30f5-23c5-462d-8e3a-a68013dd28d6 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:03.669817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c30f5-23c5-462d-8e3a-a68013dd28d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:03.691252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-482c30f5-23c5-462d-8e3a-a68013dd28d6 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:03.692895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-482c30f5-23c5-462d-8e3a-a68013dd28d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:03.778544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:03.786470 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 321/636] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:03 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:04.101444 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50bf4d-ee2a-4dc2-b843-a14501ba6033 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:04.217860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50bf4d-ee2a-4dc2-b843-a14501ba6033 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:04.258306 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095829e-fdbe-40ce-a6c8-c80fb34eecb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:04.259351 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095829e-fdbe-40ce-a6c8-c80fb34eecb0 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:04.259876 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095829e-fdbe-40ce-a6c8-c80fb34eecb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:04.260386 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5095829e-fdbe-40ce-a6c8-c80fb34eecb0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:04.272834 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5095829e-fdbe-40ce-a6c8-c80fb34eecb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:36:04.278685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-5095829e-fdbe-40ce-a6c8-c80fb34eecb0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 30 00:36:04.278685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:04.278685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:04.278685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:04.278685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:04.278685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:04.278685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:04.278685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:04.278685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:04.278685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:04.278685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:04.278685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:04.278685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:36:04.278685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:36:04.278685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:36:04.278685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:36:04.279687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:36:04.279687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:36:04.279687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:36:04.279687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:36:04.279687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:04.279687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:04.279687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:04.279687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:04.279687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:04.279687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:04.279687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:04.279687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:04.279687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:04.279687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:04.279687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:04.279687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:04.280602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:04.280602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:04.280602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:04.280602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:04.280602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:36:04.280602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:36:04.280602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:36:04.280602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:36:04.280602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:36:04.280602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:36:04.280602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 30 00:36:04.280602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:04.283445 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 316/637] 10.222.0.36 () {56 vars in 1120 bytes} [Sun Oct 30 00:36:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:04.289614 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c30f5-23c5-462d-8e3a-a68013dd28d6 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:04.418528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c30f5-23c5-462d-8e3a-a68013dd28d6 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:04.470005 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ebac3-fba6-45a6-b34b-4fbc7fe21864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:04.470928 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ebac3-fba6-45a6-b34b-4fbc7fe21864 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:04.471386 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ebac3-fba6-45a6-b34b-4fbc7fe21864 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:04.471833 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ebac3-fba6-45a6-b34b-4fbc7fe21864 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:04.472930 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ebac3-fba6-45a6-b34b-4fbc7fe21864 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:04.473405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ebac3-fba6-45a6-b34b-4fbc7fe21864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:04.474890 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ebac3-fba6-45a6-b34b-4fbc7fe21864 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:04.488109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 322/638] 10.222.0.36 () {56 vars in 1115 bytes} [Sun Oct 30 00:36:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:04.493619 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095829e-fdbe-40ce-a6c8-c80fb34eecb0 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:04.560531 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:04.605751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5095829e-fdbe-40ce-a6c8-c80fb34eecb0 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:04.643328 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f51483-d7b5-45a9-a44c-130b78ee18fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:04.643977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f51483-d7b5-45a9-a44c-130b78ee18fc None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:04.644289 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f51483-d7b5-45a9-a44c-130b78ee18fc None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:04.644630 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f51483-d7b5-45a9-a44c-130b78ee18fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:04.655800 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f51483-d7b5-45a9-a44c-130b78ee18fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:04.657307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f51483-d7b5-45a9-a44c-130b78ee18fc None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:04.674152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-f3f51483-d7b5-45a9-a44c-130b78ee18fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:04.674152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:04.674152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:04.674152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:04.674152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:04.674152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:04.674152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:04.674152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:04.674152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:04.674152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:04.674152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:04.674152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:04.674152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:36:04.674152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:36:04.674152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:36:04.674152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:36:04.675892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:04.675892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:04.675892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:04.675892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:04.675892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:04.675892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:04.675892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:04.675892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:04.675892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:04.675892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:04.675892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:04.675892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:04.675892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:04.675892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:04.675892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:04.675892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:04.675892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:36:04.677056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:36:04.677056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:04.677056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:04.677056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 317/639] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:04.681544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ebac3-fba6-45a6-b34b-4fbc7fe21864 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:04.752628 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:04.798315 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ebac3-fba6-45a6-b34b-4fbc7fe21864 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:04.817308 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b02274-7e98-421b-929b-36e285851924 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:04.817646 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b02274-7e98-421b-929b-36e285851924 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:04.817871 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b02274-7e98-421b-929b-36e285851924 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:04.819489 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 318/640] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:36:04.825481 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa873add-4407-40f6-adb0-39ef24d02103 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:04.825798 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa873add-4407-40f6-adb0-39ef24d02103 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:04.826036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa873add-4407-40f6-adb0-39ef24d02103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:04.838187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371db953-a79e-4a51-a9a4-00a046879737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:04.839844 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371db953-a79e-4a51-a9a4-00a046879737 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:04.840376 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371db953-a79e-4a51-a9a4-00a046879737 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:04.840863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371db953-a79e-4a51-a9a4-00a046879737 None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:04.841795 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371db953-a79e-4a51-a9a4-00a046879737 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:04.842312 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371db953-a79e-4a51-a9a4-00a046879737 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:04.843955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371db953-a79e-4a51-a9a4-00a046879737 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:04.845595 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-aa873add-4407-40f6-adb0-39ef24d02103 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:04.846821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-aa873add-4407-40f6-adb0-39ef24d02103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:04.858547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 323/641] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:04.865488 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371db953-a79e-4a51-a9a4-00a046879737 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:04.935968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 319/642] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:36:05.007995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371db953-a79e-4a51-a9a4-00a046879737 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:05.031080 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:05.047936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0a926-09ae-4940-a04b-ad500b4f4586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:05.048610 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0a926-09ae-4940-a04b-ad500b4f4586 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:05.048883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0a926-09ae-4940-a04b-ad500b4f4586 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:05.049153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0a926-09ae-4940-a04b-ad500b4f4586 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:05.057001 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0a926-09ae-4940-a04b-ad500b4f4586 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:05.059108 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0a926-09ae-4940-a04b-ad500b4f4586 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:05.071690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-80a0a926-09ae-4940-a04b-ad500b4f4586 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 30 00:36:05.071690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:05.071690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:05.071690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:05.071690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:05.071690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:05.071690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:05.071690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:05.071690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:05.071690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:05.071690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:05.071690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:05.071690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:36:05.071690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:36:05.071690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:36:05.071690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:36:05.072774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:05.072774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:05.072774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:36:05.072774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:36:05.072774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:36:05.072774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:36:05.072774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:05.072774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:05.072774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:05.072774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:05.072774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:05.072774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:05.072774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:05.072774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:05.072774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:05.072774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:05.072774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:05.072774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:05.073815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:05.073815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:05.073815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:36:05.073815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:36:05.073815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:36:05.073815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:36:05.073815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:36:05.073815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:36:05.073815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 30 00:36:05.073815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:05.073815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 324/643] 10.222.0.36 () {56 vars in 1117 bytes} [Sun Oct 30 00:36:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:05.078651 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa873add-4407-40f6-adb0-39ef24d02103 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:05.128504 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:05.192263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa873add-4407-40f6-adb0-39ef24d02103 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:05.220504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0a926-09ae-4940-a04b-ad500b4f4586 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:05.233681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de61509-bbd1-48a7-a58f-36e7a72a3c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:05.234311 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de61509-bbd1-48a7-a58f-36e7a72a3c3d None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:05.234591 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de61509-bbd1-48a7-a58f-36e7a72a3c3d None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:05.234866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de61509-bbd1-48a7-a58f-36e7a72a3c3d None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:05.235728 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de61509-bbd1-48a7-a58f-36e7a72a3c3d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:05.236036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de61509-bbd1-48a7-a58f-36e7a72a3c3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:05.237563 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de61509-bbd1-48a7-a58f-36e7a72a3c3d None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:05.268862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 320/644] 10.222.0.36 () {56 vars in 1149 bytes} [Sun Oct 30 00:36:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:05.269846 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de61509-bbd1-48a7-a58f-36e7a72a3c3d None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:05.380911 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0a926-09ae-4940-a04b-ad500b4f4586 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:05.396497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:05.400494 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de61509-bbd1-48a7-a58f-36e7a72a3c3d None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:05.435770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da44db-d6d7-49fc-809e-9f73f0a05376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:05.436522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da44db-d6d7-49fc-809e-9f73f0a05376 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:05.436808 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da44db-d6d7-49fc-809e-9f73f0a05376 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:05.437087 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da44db-d6d7-49fc-809e-9f73f0a05376 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:05.438156 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da44db-d6d7-49fc-809e-9f73f0a05376 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:05.440120 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da44db-d6d7-49fc-809e-9f73f0a05376 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:05.453921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75538f-401d-4bca-84e6-f0d2803745b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:05.454253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-c9da44db-d6d7-49fc-809e-9f73f0a05376 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 30 00:36:05.454253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:05.454253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:05.454253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:05.454253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:05.454253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:05.454253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:05.454253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:05.454253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 30 00:36:05.454253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:05.454253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:05.454253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:05.454253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:05.454253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:05.454253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:05.454253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:05.454253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:05.455213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:05.455213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:05.455213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:05.455213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:05.455213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:05.455213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:05.455213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:05.455213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:05.455213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:05.455213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:05.455213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:05.455213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:05.455213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:05.455213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:05.455213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:05.455213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:05.456049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:05.456049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:05.456049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:05.456049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:05.456049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:05.456049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:05.456049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:05.456049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:05.456049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:05.456049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:05.456049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:05.456049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:05.456049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 30 00:36:05.456049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:05.456049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 325/645] 10.222.0.36 () {56 vars in 1114 bytes} [Sun Oct 30 00:36:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:36:05.458251 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75538f-401d-4bca-84e6-f0d2803745b9 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:05.458762 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75538f-401d-4bca-84e6-f0d2803745b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:05.459242 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75538f-401d-4bca-84e6-f0d2803745b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:05.461556 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da44db-d6d7-49fc-809e-9f73f0a05376 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:05.472417 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75538f-401d-4bca-84e6-f0d2803745b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:05.474240 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75538f-401d-4bca-84e6-f0d2803745b9 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:05.490708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-8e75538f-401d-4bca-84e6-f0d2803745b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:05.490708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:05.490708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:05.490708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:05.490708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:05.490708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:05.490708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:05.490708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:05.490708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:05.490708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:05.490708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:05.490708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:05.490708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:36:05.490708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:36:05.490708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:36:05.490708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:36:05.491727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:05.491727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:05.491727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:05.491727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:05.491727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:05.491727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:05.491727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:05.491727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:05.491727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:05.491727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:05.491727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:05.491727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:05.491727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:05.491727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:05.491727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:05.491727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:05.491727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:36:05.492692 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:36:05.492692 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:05.492692 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:05.494732 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 321/646] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:05.503315 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75538f-401d-4bca-84e6-f0d2803745b9 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:05.529939 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:05.598040 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da44db-d6d7-49fc-809e-9f73f0a05376 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:05.648245 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75538f-401d-4bca-84e6-f0d2803745b9 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:05.649972 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96d16b-c89c-44b6-b67a-12cb13ffebba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:05.650594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96d16b-c89c-44b6-b67a-12cb13ffebba None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:05.650926 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96d16b-c89c-44b6-b67a-12cb13ffebba None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:05.651139 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96d16b-c89c-44b6-b67a-12cb13ffebba None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:05.652310 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96d16b-c89c-44b6-b67a-12cb13ffebba None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:05.653022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96d16b-c89c-44b6-b67a-12cb13ffebba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:05.654495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96d16b-c89c-44b6-b67a-12cb13ffebba None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:05.659769 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 326/647] 10.222.0.36 () {56 vars in 1114 bytes} [Sun Oct 30 00:36:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:05.661120 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4db7b-cffe-402d-99de-a361eb339873 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:05.661399 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4db7b-cffe-402d-99de-a361eb339873 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:05.661668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4db7b-cffe-402d-99de-a361eb339873 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:05.668013 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 327/648] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:05.668013 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4db7b-cffe-402d-99de-a361eb339873 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:05.697843 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3f758-4b82-4520-ae72-d3667057faf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:05.699738 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3f758-4b82-4520-ae72-d3667057faf0 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:05.700235 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3f758-4b82-4520-ae72-d3667057faf0 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:05.701021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3f758-4b82-4520-ae72-d3667057faf0 None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:05.702134 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3f758-4b82-4520-ae72-d3667057faf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:05.702595 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3f758-4b82-4520-ae72-d3667057faf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:05.704299 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3f758-4b82-4520-ae72-d3667057faf0 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:05.724703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 322/649] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:05.725997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd8413-5008-4602-b58e-19ae460dabf6 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:05.726236 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd8413-5008-4602-b58e-19ae460dabf6 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:05.726513 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd8413-5008-4602-b58e-19ae460dabf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:05.745491 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-facd8413-5008-4602-b58e-19ae460dabf6 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:05.747430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-facd8413-5008-4602-b58e-19ae460dabf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:05.755245 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:05.776072 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4db7b-cffe-402d-99de-a361eb339873 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:05.786664 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:05.814413 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f0f55-cb06-4e85-9118-2030f0724070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:05.815033 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f0f55-cb06-4e85-9118-2030f0724070 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:05.815308 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f0f55-cb06-4e85-9118-2030f0724070 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:05.815582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f0f55-cb06-4e85-9118-2030f0724070 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:05.816251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f0f55-cb06-4e85-9118-2030f0724070 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:05.816555 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f0f55-cb06-4e85-9118-2030f0724070 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:05.817948 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f0f55-cb06-4e85-9118-2030f0724070 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:05.820301 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 323/650] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:05.821162 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd8413-5008-4602-b58e-19ae460dabf6 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:05.828985 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 328/651] 10.222.0.36 () {56 vars in 1114 bytes} [Sun Oct 30 00:36:05 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:05.828985 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8592fe-1053-45bf-99b1-e4a481c61e6b None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:05.828985 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8592fe-1053-45bf-99b1-e4a481c61e6b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:05.828985 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8592fe-1053-45bf-99b1-e4a481c61e6b None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:05.828985 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 329/652] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:05.838237 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84922016-e341-43e7-ba11-078f32287100 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:05.838518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84922016-e341-43e7-ba11-078f32287100 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:05.838846 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84922016-e341-43e7-ba11-078f32287100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:05.869370 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-84922016-e341-43e7-ba11-078f32287100 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:05.874138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-84922016-e341-43e7-ba11-078f32287100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:05.977591 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd8413-5008-4602-b58e-19ae460dabf6 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:05.988519 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 330/653] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:36:06.019248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee7ba1-eb17-49d9-aa12-091f642ded80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:06.019901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee7ba1-eb17-49d9-aa12-091f642ded80 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:06.020209 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee7ba1-eb17-49d9-aa12-091f642ded80 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:06.020483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee7ba1-eb17-49d9-aa12-091f642ded80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:06.030152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee7ba1-eb17-49d9-aa12-091f642ded80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:06.031631 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee7ba1-eb17-49d9-aa12-091f642ded80 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:06.044125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-caee7ba1-eb17-49d9-aa12-091f642ded80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:36:06.044125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:06.044125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:06.044125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:06.044125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:06.044125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:06.044125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:06.044125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:06.044125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:06.044125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:06.044125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:06.044125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:06.044125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:36:06.044125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:36:06.044125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:36:06.045570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:36:06.045570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:06.045570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:06.045570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:06.045570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:06.045570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:06.045570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:06.045570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:06.045570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:06.045570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:06.045570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:06.045570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:06.045570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:06.045570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:06.045570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:06.045570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:06.045570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:06.047369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:36:06.047369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:36:06.047369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:36:06.047369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:36:06.047369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:36:06.047369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:36:06.047369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:36:06.047369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:06.047369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 324/654] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:36:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:06.051903 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84922016-e341-43e7-ba11-078f32287100 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:06.103706 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee7ba1-eb17-49d9-aa12-091f642ded80 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:06.107668 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:06.182409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84922016-e341-43e7-ba11-078f32287100 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:06.184851 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:06.232195 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee7ba1-eb17-49d9-aa12-091f642ded80 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:06.234733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791debe-125a-4922-8aae-515bf1897ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:06.236494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791debe-125a-4922-8aae-515bf1897ebc None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:06.236686 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791debe-125a-4922-8aae-515bf1897ebc None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:06.236966 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791debe-125a-4922-8aae-515bf1897ebc None admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:06.237674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791debe-125a-4922-8aae-515bf1897ebc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:06.237985 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791debe-125a-4922-8aae-515bf1897ebc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:06.239461 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791debe-125a-4922-8aae-515bf1897ebc None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:06.257287 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 331/655] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:36:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:06.264224 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791debe-125a-4922-8aae-515bf1897ebc None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:06.277630 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d51e20-a5f5-4cfd-83c7-a9bcd8570696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:06.278536 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d51e20-a5f5-4cfd-83c7-a9bcd8570696 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:06.278960 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d51e20-a5f5-4cfd-83c7-a9bcd8570696 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:06.279374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d51e20-a5f5-4cfd-83c7-a9bcd8570696 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:06.280128 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d51e20-a5f5-4cfd-83c7-a9bcd8570696 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:06.281680 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d51e20-a5f5-4cfd-83c7-a9bcd8570696 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:06.293029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-64d51e20-a5f5-4cfd-83c7-a9bcd8570696 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:36:06.293029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:06.293029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:06.293029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:06.293029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:06.293029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:06.293029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:06.293029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:06.293029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:36:06.293029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:06.293029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:06.293029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:06.293029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:06.293029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:06.293029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:06.293029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:06.293029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:06.294158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:06.294158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:06.294158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:06.294158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:06.294158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:06.294158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:06.294158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:06.294158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:06.294158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:06.294158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:06.294158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:06.294158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:06.294158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:06.294158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:06.294158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:06.294158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:06.295048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:06.295048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:06.295048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:06.295048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:06.295048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:06.295048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:06.295048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:06.295048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:06.295048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:06.295048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:06.295048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:06.295048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:06.295048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:36:06.295048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:06.295838 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 325/656] 10.222.0.36 () {56 vars in 1126 bytes} [Sun Oct 30 00:36:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:36:06.296519 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d51e20-a5f5-4cfd-83c7-a9bcd8570696 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:06.378580 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:06.394167 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791debe-125a-4922-8aae-515bf1897ebc None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:06.437794 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d51e20-a5f5-4cfd-83c7-a9bcd8570696 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:06.446910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892dca3d-35d7-491f-ae64-3724fa82ca73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:06.447577 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dca3d-35d7-491f-ae64-3724fa82ca73 None admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:06.447857 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dca3d-35d7-491f-ae64-3724fa82ca73 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:06.448154 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dca3d-35d7-491f-ae64-3724fa82ca73 None admin] PATH_INFO: `/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/8a91d3b9c4834d729211ec367e235889/roles/f74a886e30fd49cd82e0a6ed7e0f7927` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:06.450729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892dca3d-35d7-491f-ae64-3724fa82ca73 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab3a0e5fc5a344739b58ec8313639496, user_id=8a91d3b9c4834d729211ec367e235889, role_id=f74a886e30fd49cd82e0a6ed7e0f7927)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:06.452206 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892dca3d-35d7-491f-ae64-3724fa82ca73 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:06.476116 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 332/657] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:36:06 2022] PUT /identity/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/8a91d3b9c4834d729211ec367e235889/roles/f74a886e30fd49cd82e0a6ed7e0f7927 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:36:06.476116 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892dca3d-35d7-491f-ae64-3724fa82ca73 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:06.492662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453f56d-646c-47cf-aa80-880fa0119fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:06.493405 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453f56d-646c-47cf-aa80-880fa0119fbe None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:06.493793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453f56d-646c-47cf-aa80-880fa0119fbe None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:06.494253 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453f56d-646c-47cf-aa80-880fa0119fbe None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:06.495000 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453f56d-646c-47cf-aa80-880fa0119fbe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:06.497546 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453f56d-646c-47cf-aa80-880fa0119fbe None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:06.510666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:06.523588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 326/658] 10.222.0.36 () {60 vars in 1141 bytes} [Sun Oct 30 00:36:06 2022] POST /identity/v3/services => generated 269 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:36:06.616480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892dca3d-35d7-491f-ae64-3724fa82ca73 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:06.672670 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e8547-b9c0-495a-ae9e-88f5ddbace7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:06.673418 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e8547-b9c0-495a-ae9e-88f5ddbace7b None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:06.673752 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e8547-b9c0-495a-ae9e-88f5ddbace7b None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:06.674070 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e8547-b9c0-495a-ae9e-88f5ddbace7b None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:06.674742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e8547-b9c0-495a-ae9e-88f5ddbace7b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:06.675047 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e8547-b9c0-495a-ae9e-88f5ddbace7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:06.676680 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e8547-b9c0-495a-ae9e-88f5ddbace7b None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:06.682727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 333/659] 10.222.0.36 () {56 vars in 1122 bytes} [Sun Oct 30 00:36:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:06.689043 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453f56d-646c-47cf-aa80-880fa0119fbe None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:06.822745 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453f56d-646c-47cf-aa80-880fa0119fbe None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:06.858780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:06.872566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22188f5a-ba89-4751-90af-9ac5ab66cadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:06.873512 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22188f5a-ba89-4751-90af-9ac5ab66cadd None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:06.873967 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22188f5a-ba89-4751-90af-9ac5ab66cadd None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:06.874435 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22188f5a-ba89-4751-90af-9ac5ab66cadd None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:06.875525 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22188f5a-ba89-4751-90af-9ac5ab66cadd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:06.876063 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22188f5a-ba89-4751-90af-9ac5ab66cadd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:06.877871 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22188f5a-ba89-4751-90af-9ac5ab66cadd None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:06.883419 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 327/660] 10.222.0.36 () {56 vars in 1122 bytes} [Sun Oct 30 00:36:06 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:08.258236 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17720fd-5e78-4cc4-9270-df50fd82eab0 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:08.258236 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17720fd-5e78-4cc4-9270-df50fd82eab0 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:08.258236 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17720fd-5e78-4cc4-9270-df50fd82eab0 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:08.259247 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 334/661] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:08.266362 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf21fe-2613-471e-99a1-2259292ff18b None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:08.267252 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf21fe-2613-471e-99a1-2259292ff18b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:08.267701 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf21fe-2613-471e-99a1-2259292ff18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:08.287405 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-b8cf21fe-2613-471e-99a1-2259292ff18b None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:08.288912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-b8cf21fe-2613-471e-99a1-2259292ff18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:08.366771 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 328/662] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:08 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:36:08.640062 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17720fd-5e78-4cc4-9270-df50fd82eab0 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:08.736263 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:08.756632 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17720fd-5e78-4cc4-9270-df50fd82eab0 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:08.795458 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ddff8-eb9f-4149-a8ed-12322496c17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:08.796114 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ddff8-eb9f-4149-a8ed-12322496c17a None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:08.796684 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ddff8-eb9f-4149-a8ed-12322496c17a None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:08.796965 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ddff8-eb9f-4149-a8ed-12322496c17a None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:08.797661 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ddff8-eb9f-4149-a8ed-12322496c17a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:08.799097 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ddff8-eb9f-4149-a8ed-12322496c17a None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:08.820035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 335/663] 10.222.0.36 () {60 vars in 1141 bytes} [Sun Oct 30 00:36:08 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:36:08.956324 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f1ec7-d645-4f52-9f14-107f5d2b1ebe None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:08.956814 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f1ec7-d645-4f52-9f14-107f5d2b1ebe None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:08.957172 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f1ec7-d645-4f52-9f14-107f5d2b1ebe None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:08.959115 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 329/664] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:08.965726 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1170a5f-1c42-4650-a143-385883275c41 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:08.966008 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1170a5f-1c42-4650-a143-385883275c41 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:08.966449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1170a5f-1c42-4650-a143-385883275c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:08.987997 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-d1170a5f-1c42-4650-a143-385883275c41 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:08.989253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d1170a5f-1c42-4650-a143-385883275c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:09.070579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e778604-5342-4959-8dad-636892afab48 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:09.071090 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e778604-5342-4959-8dad-636892afab48 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:09.071574 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e778604-5342-4959-8dad-636892afab48 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:09.073247 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 330/665] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:09.079119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 336/666] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:08 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:09.079696 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd061f-0255-42f8-b461-04f6a1cd97b8 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:09.080046 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd061f-0255-42f8-b461-04f6a1cd97b8 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:09.080372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd061f-0255-42f8-b461-04f6a1cd97b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:09.104562 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-17fd061f-0255-42f8-b461-04f6a1cd97b8 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:09.107721 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-17fd061f-0255-42f8-b461-04f6a1cd97b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:09.193151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 331/667] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:09 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:36:09.359216 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1170a5f-1c42-4650-a143-385883275c41 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:09.414321 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508e99b-a9fa-4959-87c9-7a72a416c4d3 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:09.415124 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508e99b-a9fa-4959-87c9-7a72a416c4d3 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:09.415124 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508e99b-a9fa-4959-87c9-7a72a416c4d3 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:09.416741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 332/668] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:09.424566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68732377-f9c6-4096-a25d-f5071b9bc9af None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:09.424925 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68732377-f9c6-4096-a25d-f5071b9bc9af None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:09.425241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68732377-f9c6-4096-a25d-f5071b9bc9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:09.432009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:09.451666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-68732377-f9c6-4096-a25d-f5071b9bc9af None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:09.453412 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-68732377-f9c6-4096-a25d-f5071b9bc9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:09.470346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1170a5f-1c42-4650-a143-385883275c41 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:09.509910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac066984-17cd-43fb-b6af-8f25a0612134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:09.510562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac066984-17cd-43fb-b6af-8f25a0612134 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:09.510921 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac066984-17cd-43fb-b6af-8f25a0612134 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:09.511105 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac066984-17cd-43fb-b6af-8f25a0612134 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:09.517528 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:09.523797 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac066984-17cd-43fb-b6af-8f25a0612134 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:36:09.529460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-ac066984-17cd-43fb-b6af-8f25a0612134 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 30 00:36:09.529460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:09.529460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:09.529460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:09.529460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:09.529460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:09.529460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:09.529460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:09.529460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:09.529460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:09.529460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:09.529460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:09.529460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:36:09.529460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:36:09.529460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:36:09.529460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:36:09.530673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:36:09.530673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:36:09.530673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:36:09.530673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:36:09.530673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:09.530673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:09.530673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:09.530673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:09.530673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:09.530673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:09.530673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:09.530673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:09.530673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:09.530673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:09.530673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:09.530673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:09.531653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:09.531653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:09.531653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:09.531653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:09.531653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:36:09.531653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:36:09.531653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:36:09.531653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:36:09.531653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:36:09.531653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:36:09.531653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 30 00:36:09.531653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:09.531653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 337/669] 10.222.0.36 () {56 vars in 1120 bytes} [Sun Oct 30 00:36:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:09.532522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac066984-17cd-43fb-b6af-8f25a0612134 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:09.555603 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 333/670] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:09 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:36:09.557656 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68732377-f9c6-4096-a25d-f5071b9bc9af None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:09.658791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac066984-17cd-43fb-b6af-8f25a0612134 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:09.696636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:09.718978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3754f-8191-4ff4-b4e4-7c053b0085b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:09.721347 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3754f-8191-4ff4-b4e4-7c053b0085b4 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:09.721987 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3754f-8191-4ff4-b4e4-7c053b0085b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:09.722469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3754f-8191-4ff4-b4e4-7c053b0085b4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:09.723497 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f3754f-8191-4ff4-b4e4-7c053b0085b4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:09.725462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f3754f-8191-4ff4-b4e4-7c053b0085b4 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:09.725692 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68732377-f9c6-4096-a25d-f5071b9bc9af None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:09.739242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-b5f3754f-8191-4ff4-b4e4-7c053b0085b4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 30 00:36:09.739242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:09.739242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:09.739242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:09.739242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:09.739242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:09.739242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:09.739242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:09.739242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 30 00:36:09.739242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:09.739242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:09.739242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:09.739242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:09.739242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:09.739242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:09.739242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:09.740591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:09.740591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:09.740591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:09.740591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:09.740591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:09.740591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:09.740591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:09.740591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:09.740591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:09.740591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:09.740591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:09.740591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:09.740591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:09.740591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:09.740591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:09.740591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:09.740591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:09.741870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:09.741870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:09.741870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:09.741870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:09.741870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:09.741870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:09.741870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:09.741870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:09.741870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:09.741870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:09.741870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:09.741870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:09.741870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 30 00:36:09.741870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:09.744259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 338/671] 10.222.0.36 () {56 vars in 1132 bytes} [Sun Oct 30 00:36:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:09.747983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3754f-8191-4ff4-b4e4-7c053b0085b4 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:09.773603 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf58773-49b6-4e23-b81f-5753049f3fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:09.774229 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf58773-49b6-4e23-b81f-5753049f3fb4 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:09.774507 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf58773-49b6-4e23-b81f-5753049f3fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:09.774778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf58773-49b6-4e23-b81f-5753049f3fb4 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:09.775451 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf58773-49b6-4e23-b81f-5753049f3fb4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:09.775812 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf58773-49b6-4e23-b81f-5753049f3fb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:09.777885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf58773-49b6-4e23-b81f-5753049f3fb4 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:09.791355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 334/672] 10.222.0.36 () {56 vars in 1115 bytes} [Sun Oct 30 00:36:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:09.802482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf58773-49b6-4e23-b81f-5753049f3fb4 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:09.851566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:09.884549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3754f-8191-4ff4-b4e4-7c053b0085b4 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:09.932468 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf58773-49b6-4e23-b81f-5753049f3fb4 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:09.934428 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f9444-93a4-4477-bf1c-732379b4c27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:09.935395 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f9444-93a4-4477-bf1c-732379b4c27c None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:09.935787 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f9444-93a4-4477-bf1c-732379b4c27c None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:09.936278 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f9444-93a4-4477-bf1c-732379b4c27c None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:09.937333 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f9444-93a4-4477-bf1c-732379b4c27c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:09.937771 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f9444-93a4-4477-bf1c-732379b4c27c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:09.940062 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f9444-93a4-4477-bf1c-732379b4c27c None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:09.952587 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 339/673] 10.222.0.36 () {56 vars in 1126 bytes} [Sun Oct 30 00:36:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:09.952587 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f9444-93a4-4477-bf1c-732379b4c27c None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:09.985047 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7bb44b-d8bd-4682-a027-bb68d14ae11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:09.985756 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bb44b-d8bd-4682-a027-bb68d14ae11d None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:09.986065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bb44b-d8bd-4682-a027-bb68d14ae11d None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:09.986350 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bb44b-d8bd-4682-a027-bb68d14ae11d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:09.998431 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7bb44b-d8bd-4682-a027-bb68d14ae11d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:09.999905 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7bb44b-d8bd-4682-a027-bb68d14ae11d None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:10.015541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-ef7bb44b-d8bd-4682-a027-bb68d14ae11d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:10.015541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:10.015541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:10.015541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:10.015541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:10.015541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:10.015541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:10.015541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:10.015541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:10.015541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:10.015541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:10.015541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:10.015541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:36:10.015541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:36:10.015541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:36:10.015541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:36:10.016901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:10.016901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:10.016901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:10.016901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:10.016901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:10.016901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:10.016901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:10.016901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:10.016901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:10.016901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:10.016901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:10.016901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:10.016901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:10.016901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:10.016901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:10.016901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:10.016901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:36:10.017830 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:36:10.017830 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:10.017830 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:10.018002 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:10.021622 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 335/674] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:36:10.022562 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7bb44b-d8bd-4682-a027-bb68d14ae11d None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:10.075138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f9444-93a4-4477-bf1c-732379b4c27c None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:10.116589 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f888b-e200-47da-94fc-a296b5dc8a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:10.117510 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f888b-e200-47da-94fc-a296b5dc8a6b None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:10.117970 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f888b-e200-47da-94fc-a296b5dc8a6b None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:10.118418 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f888b-e200-47da-94fc-a296b5dc8a6b None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:10.119492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f888b-e200-47da-94fc-a296b5dc8a6b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:10.121125 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f888b-e200-47da-94fc-a296b5dc8a6b None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:10.137635 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 340/675] 10.222.0.36 () {60 vars in 1141 bytes} [Sun Oct 30 00:36:09 2022] POST /identity/v3/services => generated 241 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:36:10.139044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f888b-e200-47da-94fc-a296b5dc8a6b None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:10.155103 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:10.166647 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7bb44b-d8bd-4682-a027-bb68d14ae11d None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:10.231865 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3b91f-d5ee-485f-a615-71801adcd551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:10.232441 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3b91f-d5ee-485f-a615-71801adcd551 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:10.233273 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3b91f-d5ee-485f-a615-71801adcd551 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:10.233616 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3b91f-d5ee-485f-a615-71801adcd551 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:10.234588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3b91f-d5ee-485f-a615-71801adcd551 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:10.234894 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3b91f-d5ee-485f-a615-71801adcd551 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:10.236455 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3b91f-d5ee-485f-a615-71801adcd551 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:10.244878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 336/676] 10.222.0.36 () {56 vars in 1126 bytes} [Sun Oct 30 00:36:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:10.251329 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3b91f-d5ee-485f-a615-71801adcd551 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:10.301984 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f888b-e200-47da-94fc-a296b5dc8a6b None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:10.323596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:10.348637 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f72e7c-a3fc-4182-9b64-708a32b54470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:10.349916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f72e7c-a3fc-4182-9b64-708a32b54470 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:10.350199 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f72e7c-a3fc-4182-9b64-708a32b54470 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:10.350644 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f72e7c-a3fc-4182-9b64-708a32b54470 None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:10.351634 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f72e7c-a3fc-4182-9b64-708a32b54470 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:10.352054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f72e7c-a3fc-4182-9b64-708a32b54470 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:10.353483 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f72e7c-a3fc-4182-9b64-708a32b54470 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:10.367614 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 341/677] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:10.375277 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f72e7c-a3fc-4182-9b64-708a32b54470 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:10.416370 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3b91f-d5ee-485f-a615-71801adcd551 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:10.458936 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42cef6a-6ccc-4ff3-8c73-5623325b8e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:10.459774 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cef6a-6ccc-4ff3-8c73-5623325b8e29 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:10.459977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cef6a-6ccc-4ff3-8c73-5623325b8e29 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:10.460246 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cef6a-6ccc-4ff3-8c73-5623325b8e29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:10.461052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42cef6a-6ccc-4ff3-8c73-5623325b8e29 None admin] RBAC: Adding query filter params (interface=public, service_id=8746772db0c84926b31eea8c4d82c634, region_id=RegionOne) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:10.461257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42cef6a-6ccc-4ff3-8c73-5623325b8e29 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:10.462643 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42cef6a-6ccc-4ff3-8c73-5623325b8e29 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:10.469361 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 337/678] 10.222.0.36 () {56 vars in 1251 bytes} [Sun Oct 30 00:36:10 2022] GET /identity/v3/endpoints?service_id=8746772db0c84926b31eea8c4d82c634&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:10.475860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42cef6a-6ccc-4ff3-8c73-5623325b8e29 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:10.499693 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:10.517551 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f72e7c-a3fc-4182-9b64-708a32b54470 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:10.555899 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95509e-7bd1-4da4-bdfd-bf151cd21ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:10.556619 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95509e-7bd1-4da4-bdfd-bf151cd21ac3 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:10.556908 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95509e-7bd1-4da4-bdfd-bf151cd21ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:10.557169 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95509e-7bd1-4da4-bdfd-bf151cd21ac3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:10.565990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95509e-7bd1-4da4-bdfd-bf151cd21ac3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:10.568549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95509e-7bd1-4da4-bdfd-bf151cd21ac3 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:10.581538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-3f95509e-7bd1-4da4-bdfd-bf151cd21ac3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 30 00:36:10.581538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:10.581538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:10.581538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:10.581538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:10.581538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:10.581538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:10.581538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:10.581538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:10.581538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:10.581538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:10.581538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:10.581538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:36:10.581538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:36:10.581538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:36:10.581538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:36:10.582652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:10.582652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:10.582652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:36:10.582652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:36:10.582652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:36:10.582652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:36:10.582652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:10.582652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:10.582652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:10.582652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:10.582652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:10.582652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:10.582652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:10.582652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:10.582652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:10.582652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:10.582652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:10.582652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:10.583597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:10.583597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:10.583597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:36:10.583597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:36:10.583597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:36:10.583597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:36:10.583597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:36:10.583597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:36:10.583597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 30 00:36:10.583597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:10.587911 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 342/679] 10.222.0.36 () {56 vars in 1117 bytes} [Sun Oct 30 00:36:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:10.588986 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95509e-7bd1-4da4-bdfd-bf151cd21ac3 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:10.607793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42cef6a-6ccc-4ff3-8c73-5623325b8e29 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:10.629297 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:10.650679 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa61c1f-799b-499f-b31a-8d2dc1905bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:10.652062 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa61c1f-799b-499f-b31a-8d2dc1905bbb None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:10.652867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa61c1f-799b-499f-b31a-8d2dc1905bbb None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:10.653496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa61c1f-799b-499f-b31a-8d2dc1905bbb None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:10.654262 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa61c1f-799b-499f-b31a-8d2dc1905bbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:10.655793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa61c1f-799b-499f-b31a-8d2dc1905bbb None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:10.663637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 338/680] 10.222.0.36 () {56 vars in 1087 bytes} [Sun Oct 30 00:36:10 2022] GET /identity/v3/services => generated 1497 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:36:10.719704 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95509e-7bd1-4da4-bdfd-bf151cd21ac3 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:10.763108 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1cc24-4350-42ba-b7e2-c704774070e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:10.764160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1cc24-4350-42ba-b7e2-c704774070e4 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:10.764726 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1cc24-4350-42ba-b7e2-c704774070e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:10.765191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1cc24-4350-42ba-b7e2-c704774070e4 None admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:10.766302 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1cc24-4350-42ba-b7e2-c704774070e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:10.766876 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1cc24-4350-42ba-b7e2-c704774070e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:10.768580 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1cc24-4350-42ba-b7e2-c704774070e4 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:10.790334 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 343/681] 10.222.0.36 () {56 vars in 1149 bytes} [Sun Oct 30 00:36:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:10.800874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa61c1f-799b-499f-b31a-8d2dc1905bbb None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:10.901243 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:10.924861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa61c1f-799b-499f-b31a-8d2dc1905bbb None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:10.962865 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066cc5ee-43da-4276-ad63-a341935121d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:10.963493 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc5ee-43da-4276-ad63-a341935121d5 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:10.963774 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc5ee-43da-4276-ad63-a341935121d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:10.964048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066cc5ee-43da-4276-ad63-a341935121d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:11.047374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cc5ee-43da-4276-ad63-a341935121d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:11.048859 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066cc5ee-43da-4276-ad63-a341935121d5 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:11.140985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:11.169403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 339/682] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:36:10 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 30 00:36:11.175494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1cc24-4350-42ba-b7e2-c704774070e4 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:11.278753 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19eb642-3219-412a-a769-005a8fee408a None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:11.279088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19eb642-3219-412a-a769-005a8fee408a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:11.279363 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19eb642-3219-412a-a769-005a8fee408a None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:11.282292 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 340/683] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:36:11.286908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959631a7-003f-42e0-8ddb-679ba5107a72 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:11.287194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959631a7-003f-42e0-8ddb-679ba5107a72 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:11.287527 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959631a7-003f-42e0-8ddb-679ba5107a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:11.298286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:11.306528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1cc24-4350-42ba-b7e2-c704774070e4 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:11.310800 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-959631a7-003f-42e0-8ddb-679ba5107a72 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:11.312194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-959631a7-003f-42e0-8ddb-679ba5107a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:11.352520 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c257bf3-72f6-4c6f-912e-197b60ed3e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:11.353168 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c257bf3-72f6-4c6f-912e-197b60ed3e82 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:11.353448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c257bf3-72f6-4c6f-912e-197b60ed3e82 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:11.353727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c257bf3-72f6-4c6f-912e-197b60ed3e82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:11.368057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c257bf3-72f6-4c6f-912e-197b60ed3e82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:11.370147 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c257bf3-72f6-4c6f-912e-197b60ed3e82 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:11.397183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-0c257bf3-72f6-4c6f-912e-197b60ed3e82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:11.397183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:11.397183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:11.397183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:11.397183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:11.397183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:11.397183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:11.397183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:11.397183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:11.397183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:11.397183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:11.397183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:11.397183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:36:11.397183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:36:11.397183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:36:11.397183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:36:11.398274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:11.398274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:11.398274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:11.398274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:11.398274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:11.398274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:11.398274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:11.398274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:11.398274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:11.398274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:11.398274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:11.398274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:11.398274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:11.398274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:11.398274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:11.398274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:11.398274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:36:11.399648 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:36:11.399648 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:11.399648 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:11.399648 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 344/684] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:11.408152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c257bf3-72f6-4c6f-912e-197b60ed3e82 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:11.408443 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 341/685] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:36:11.533745 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c257bf3-72f6-4c6f-912e-197b60ed3e82 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:11.579405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b3fae-b88a-4d05-b9d7-6f867d7281c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:11.580302 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b3fae-b88a-4d05-b9d7-6f867d7281c5 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:11.580741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b3fae-b88a-4d05-b9d7-6f867d7281c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:11.581211 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b3fae-b88a-4d05-b9d7-6f867d7281c5 None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:11.582020 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b3fae-b88a-4d05-b9d7-6f867d7281c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:11.582485 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b3fae-b88a-4d05-b9d7-6f867d7281c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:11.584261 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b3fae-b88a-4d05-b9d7-6f867d7281c5 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:11.598717 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 345/686] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:11.605426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959631a7-003f-42e0-8ddb-679ba5107a72 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:11.722571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b3fae-b88a-4d05-b9d7-6f867d7281c5 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:11.732675 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:11.733668 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959631a7-003f-42e0-8ddb-679ba5107a72 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:11.751091 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:11.776604 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75a570-8c6f-4068-8a81-de9a038b5fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:11.778148 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75a570-8c6f-4068-8a81-de9a038b5fc7 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:11.778767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75a570-8c6f-4068-8a81-de9a038b5fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:11.779178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75a570-8c6f-4068-8a81-de9a038b5fc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:11.788224 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75a570-8c6f-4068-8a81-de9a038b5fc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:11.790075 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75a570-8c6f-4068-8a81-de9a038b5fc7 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:11.805448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-8b75a570-8c6f-4068-8a81-de9a038b5fc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:36:11.805448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:11.805448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:11.805448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:11.805448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:11.805448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:11.805448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:11.805448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:11.805448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:11.805448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:11.805448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:11.805448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:11.805448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:36:11.805448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:36:11.805448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:36:11.806467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:36:11.806467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:11.806467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:11.806467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:11.806467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:11.806467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:11.806467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:11.806467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:11.806467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:11.806467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:11.806467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:11.806467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:11.806467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:11.806467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:11.806467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:11.806467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:11.806467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:11.807393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:36:11.807393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:36:11.807393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:36:11.807393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:36:11.807393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:36:11.807393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:36:11.807393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:36:11.807393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:11.807393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 342/687] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:36:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:11.814836 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75a570-8c6f-4068-8a81-de9a038b5fc7 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:11.878850 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b3fae-b88a-4d05-b9d7-6f867d7281c5 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:11.936707 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6f123-419a-4c1b-8180-bf94cd7f3341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:11.937360 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6f123-419a-4c1b-8180-bf94cd7f3341 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:11.937637 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6f123-419a-4c1b-8180-bf94cd7f3341 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:11.937909 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6f123-419a-4c1b-8180-bf94cd7f3341 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:11.938632 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6f123-419a-4c1b-8180-bf94cd7f3341 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:11.940061 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6f123-419a-4c1b-8180-bf94cd7f3341 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:11.950147 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75a570-8c6f-4068-8a81-de9a038b5fc7 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:11.954371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-89b6f123-419a-4c1b-8180-bf94cd7f3341 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 30 00:36:11.954371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:11.954371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:11.954371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:11.954371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:11.954371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:11.954371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:11.954371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:11.954371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 30 00:36:11.954371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:11.954371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:11.954371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:11.954371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:11.954371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:11.954371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:11.954371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:11.954371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:11.955438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:11.955438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:11.955438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:11.955438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:11.955438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:11.955438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:11.955438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:11.955438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:11.955438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:11.955438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:11.955438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:11.955438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:11.955438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:11.955438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:11.955438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:11.955438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:11.956766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:11.956766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:11.956766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:11.956766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:11.956766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:11.956766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:11.956766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:11.956766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:11.956766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:11.956766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:11.956766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:11.956766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:11.956766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 30 00:36:11.956766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:11.962052 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 346/688] 10.222.0.36 () {56 vars in 1114 bytes} [Sun Oct 30 00:36:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:36:11.963580 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6f123-419a-4c1b-8180-bf94cd7f3341 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:11.996019 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6413d-e356-4731-abc3-31041af9ae0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:11.997159 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6413d-e356-4731-abc3-31041af9ae0f None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:11.997651 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6413d-e356-4731-abc3-31041af9ae0f None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:11.998124 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba6413d-e356-4731-abc3-31041af9ae0f None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:11.999036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6413d-e356-4731-abc3-31041af9ae0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:11.999850 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6413d-e356-4731-abc3-31041af9ae0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:12.002405 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba6413d-e356-4731-abc3-31041af9ae0f None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:12.021994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 343/689] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:36:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:12.022742 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6413d-e356-4731-abc3-31041af9ae0f None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:12.061314 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:12.063488 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:12.111739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6f123-419a-4c1b-8180-bf94cd7f3341 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:12.155841 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7174a-52f9-4ec1-8429-9ab26ae91d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:12.156755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7174a-52f9-4ec1-8429-9ab26ae91d1a None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:12.157236 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7174a-52f9-4ec1-8429-9ab26ae91d1a None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:12.157691 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7174a-52f9-4ec1-8429-9ab26ae91d1a None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:12.158537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7174a-52f9-4ec1-8429-9ab26ae91d1a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:12.159013 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7174a-52f9-4ec1-8429-9ab26ae91d1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:12.159366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba6413d-e356-4731-abc3-31041af9ae0f None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:12.160835 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7174a-52f9-4ec1-8429-9ab26ae91d1a None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:12.172180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 347/690] 10.222.0.36 () {56 vars in 1114 bytes} [Sun Oct 30 00:36:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:12.174411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7174a-52f9-4ec1-8429-9ab26ae91d1a None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:12.206249 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51788736-e0e9-4f63-abc2-ec2cbfbc1617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:12.207029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51788736-e0e9-4f63-abc2-ec2cbfbc1617 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:12.207384 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51788736-e0e9-4f63-abc2-ec2cbfbc1617 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:12.207725 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51788736-e0e9-4f63-abc2-ec2cbfbc1617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:12.209215 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51788736-e0e9-4f63-abc2-ec2cbfbc1617 None admin] RBAC: Adding query filter params (role.id=f74a886e30fd49cd82e0a6ed7e0f7927, scope.project.id=ab3a0e5fc5a344739b58ec8313639496, user.id=8a91d3b9c4834d729211ec367e235889) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:12.209606 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51788736-e0e9-4f63-abc2-ec2cbfbc1617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:12.211449 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51788736-e0e9-4f63-abc2-ec2cbfbc1617 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:12.220065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 344/691] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:36:12 2022] GET /identity/v3/role_assignments?user.id=8a91d3b9c4834d729211ec367e235889&scope.project.id=ab3a0e5fc5a344739b58ec8313639496&role.id=f74a886e30fd49cd82e0a6ed7e0f7927 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:12.327881 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7174a-52f9-4ec1-8429-9ab26ae91d1a None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:12.372344 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ecb13-f8fa-4cb5-a80c-ebe2ab089c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:12.373742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ecb13-f8fa-4cb5-a80c-ebe2ab089c60 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:12.374226 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ecb13-f8fa-4cb5-a80c-ebe2ab089c60 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:12.374675 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ecb13-f8fa-4cb5-a80c-ebe2ab089c60 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:12.375516 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ecb13-f8fa-4cb5-a80c-ebe2ab089c60 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:12.376006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ecb13-f8fa-4cb5-a80c-ebe2ab089c60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:12.377664 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ecb13-f8fa-4cb5-a80c-ebe2ab089c60 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:12.379550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:12.392298 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 348/692] 10.222.0.36 () {56 vars in 1114 bytes} [Sun Oct 30 00:36:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:12.394468 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51788736-e0e9-4f63-abc2-ec2cbfbc1617 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:12.517794 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:12.526152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51788736-e0e9-4f63-abc2-ec2cbfbc1617 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:12.568036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715afc42-2522-4ccd-ad10-8b987695d39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:12.569192 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715afc42-2522-4ccd-ad10-8b987695d39c None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:12.569685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715afc42-2522-4ccd-ad10-8b987695d39c None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:12.570159 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715afc42-2522-4ccd-ad10-8b987695d39c None admin] PATH_INFO: `/v3/endpoints` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:12.571027 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715afc42-2522-4ccd-ad10-8b987695d39c None admin] RBAC: Adding query filter params (interface=public, service_id=9ad7c2b05008498aaffeb04860c80957, region_id=RegionOne) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:12.571339 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715afc42-2522-4ccd-ad10-8b987695d39c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:12.572753 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715afc42-2522-4ccd-ad10-8b987695d39c None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:12.581339 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 345/693] 10.222.0.36 () {56 vars in 1251 bytes} [Sun Oct 30 00:36:12 2022] GET /identity/v3/endpoints?service_id=9ad7c2b05008498aaffeb04860c80957&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:12.587194 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ecb13-f8fa-4cb5-a80c-ebe2ab089c60 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:12.593734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f5c3f-4bd7-4d9c-b736-e89f55c4b6d2 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:12.594033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f5c3f-4bd7-4d9c-b736-e89f55c4b6d2 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:12.594301 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f5c3f-4bd7-4d9c-b736-e89f55c4b6d2 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:12.595906 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 346/694] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:36:12.603080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b0738-e881-4849-aa5a-3efe8a7c59f2 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:12.603386 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b0738-e881-4849-aa5a-3efe8a7c59f2 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:12.603625 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b0738-e881-4849-aa5a-3efe8a7c59f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:12.625285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-467b0738-e881-4849-aa5a-3efe8a7c59f2 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:12.627906 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-467b0738-e881-4849-aa5a-3efe8a7c59f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:12.720511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 347/695] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:36:12.727397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ecb13-f8fa-4cb5-a80c-ebe2ab089c60 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:12.771564 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38948750-0ca3-45df-a486-72e1b0d498b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:12.772445 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38948750-0ca3-45df-a486-72e1b0d498b2 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:12.772893 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38948750-0ca3-45df-a486-72e1b0d498b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:12.773335 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38948750-0ca3-45df-a486-72e1b0d498b2 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:12.774093 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38948750-0ca3-45df-a486-72e1b0d498b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:12.775634 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38948750-0ca3-45df-a486-72e1b0d498b2 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:12.785016 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 349/696] 10.222.0.36 () {56 vars in 1087 bytes} [Sun Oct 30 00:36:12 2022] GET /identity/v3/services => generated 1497 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:36:13.020123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b0738-e881-4849-aa5a-3efe8a7c59f2 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:13.133015 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:13.144397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd9a54-01d3-4998-b02a-ff95578d62e7 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:13.144832 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd9a54-01d3-4998-b02a-ff95578d62e7 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:13.145331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd9a54-01d3-4998-b02a-ff95578d62e7 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:13.147126 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 350/697] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:13.147939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b0738-e881-4849-aa5a-3efe8a7c59f2 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:13.153712 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0a3dc-d2a7-459d-982d-fac36c0fc0d5 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:13.153998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0a3dc-d2a7-459d-982d-fac36c0fc0d5 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:13.154277 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0a3dc-d2a7-459d-982d-fac36c0fc0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:13.175960 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-62f0a3dc-d2a7-459d-982d-fac36c0fc0d5 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:13.177578 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-62f0a3dc-d2a7-459d-982d-fac36c0fc0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:13.191634 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b94e591-ce47-4841-94b2-626b9625c04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:13.192656 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94e591-ce47-4841-94b2-626b9625c04e None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:13.193244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94e591-ce47-4841-94b2-626b9625c04e None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:13.193674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94e591-ce47-4841-94b2-626b9625c04e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:13.194859 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b94e591-ce47-4841-94b2-626b9625c04e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:13.196457 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b94e591-ce47-4841-94b2-626b9625c04e None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:13.207526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-9b94e591-ce47-4841-94b2-626b9625c04e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:36:13.207526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:13.207526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:13.207526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:13.207526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:13.207526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:13.207526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:13.207526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:13.207526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:36:13.207526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:13.207526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:13.207526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:13.207526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:13.207526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:13.207526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:13.207526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:13.207526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:13.208751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:13.208751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:13.208751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:13.208751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:13.208751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:13.208751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:13.208751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:13.208751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:13.208751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:13.208751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:13.208751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:13.208751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:13.208751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:13.208751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:13.208751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:13.208751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:13.209817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:13.209817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:13.209817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:13.209817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:13.209817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:13.209817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:13.209817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:13.209817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:13.209817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:13.209817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:13.209817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:13.209817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:13.209817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:36:13.209817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:13.209817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 348/698] 10.222.0.36 () {56 vars in 1126 bytes} [Sun Oct 30 00:36:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:13.216103 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b94e591-ce47-4841-94b2-626b9625c04e None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:13.259090 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:13.278674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 351/699] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:13.340433 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b94e591-ce47-4841-94b2-626b9625c04e None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:13.377618 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca22ee-ac50-46e2-a400-68433ef04447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:13.378309 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca22ee-ac50-46e2-a400-68433ef04447 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:13.378590 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca22ee-ac50-46e2-a400-68433ef04447 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:13.378873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca22ee-ac50-46e2-a400-68433ef04447 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:13.379494 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca22ee-ac50-46e2-a400-68433ef04447 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:13.379812 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca22ee-ac50-46e2-a400-68433ef04447 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:13.381228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca22ee-ac50-46e2-a400-68433ef04447 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:13.387432 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 349/700] 10.222.0.36 () {56 vars in 1122 bytes} [Sun Oct 30 00:36:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:13.395204 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0a3dc-d2a7-459d-982d-fac36c0fc0d5 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:13.510943 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0a3dc-d2a7-459d-982d-fac36c0fc0d5 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:13.549029 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ffac8-cdc1-4f3e-a61f-59a47d6ebd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:13.549658 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ffac8-cdc1-4f3e-a61f-59a47d6ebd07 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:13.549938 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ffac8-cdc1-4f3e-a61f-59a47d6ebd07 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:13.550211 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ffac8-cdc1-4f3e-a61f-59a47d6ebd07 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:13.550853 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ffac8-cdc1-4f3e-a61f-59a47d6ebd07 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:13.551155 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ffac8-cdc1-4f3e-a61f-59a47d6ebd07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:13.552665 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ffac8-cdc1-4f3e-a61f-59a47d6ebd07 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:13.559140 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 352/701] 10.222.0.36 () {56 vars in 1122 bytes} [Sun Oct 30 00:36:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:13.565108 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca22ee-ac50-46e2-a400-68433ef04447 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:13.586163 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:13.598591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ffac8-cdc1-4f3e-a61f-59a47d6ebd07 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:13.674397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:13.686892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca22ee-ac50-46e2-a400-68433ef04447 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:13.725512 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094f736-cae0-42a3-a8ea-86fd3ecab232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:13.726396 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094f736-cae0-42a3-a8ea-86fd3ecab232 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:13.726673 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094f736-cae0-42a3-a8ea-86fd3ecab232 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:13.726942 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094f736-cae0-42a3-a8ea-86fd3ecab232 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:13.727652 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094f736-cae0-42a3-a8ea-86fd3ecab232 None admin] RBAC: Adding query filter params (interface=public, service_id=3098d10f6fd8491b8a261bf0827e32ea, region_id=RegionOne) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:13.727956 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094f736-cae0-42a3-a8ea-86fd3ecab232 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:13.729172 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ffac8-cdc1-4f3e-a61f-59a47d6ebd07 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:13.729375 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094f736-cae0-42a3-a8ea-86fd3ecab232 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:13.740592 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 350/702] 10.222.0.36 () {56 vars in 1251 bytes} [Sun Oct 30 00:36:13 2022] GET /identity/v3/endpoints?service_id=3098d10f6fd8491b8a261bf0827e32ea&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:13.743230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094f736-cae0-42a3-a8ea-86fd3ecab232 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:13.776678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42f693-1c8f-4f20-b247-11ed13d4cfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:13.778001 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42f693-1c8f-4f20-b247-11ed13d4cfa2 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:13.778397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42f693-1c8f-4f20-b247-11ed13d4cfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:13.778677 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42f693-1c8f-4f20-b247-11ed13d4cfa2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:13.779266 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42f693-1c8f-4f20-b247-11ed13d4cfa2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:13.780878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42f693-1c8f-4f20-b247-11ed13d4cfa2 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:13.793764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-ad42f693-1c8f-4f20-b247-11ed13d4cfa2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 30 00:36:13.793764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:13.793764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:13.793764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:13.793764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:13.793764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:13.793764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:13.793764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:13.793764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 30 00:36:13.793764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:13.793764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:13.793764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:13.793764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:13.793764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:13.793764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:13.793764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:13.794927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:13.794927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:13.794927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:13.794927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:13.794927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:13.794927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:13.794927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:13.794927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:13.794927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:13.794927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:13.794927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:13.794927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:13.794927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:13.794927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:13.794927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:13.794927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:13.794927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:13.795861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:13.795861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:13.795861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:13.795861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:13.795861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:13.795861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:13.795861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:13.795861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:13.795861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:13.795861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:13.795861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:13.795861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:13.795861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 30 00:36:13.795861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:13.795861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 353/703] 10.222.0.36 () {56 vars in 1132 bytes} [Sun Oct 30 00:36:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:36:13.801977 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42f693-1c8f-4f20-b247-11ed13d4cfa2 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:13.860807 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:13.884773 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094f736-cae0-42a3-a8ea-86fd3ecab232 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:13.925410 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311abb6-18dc-47bf-9d3b-95d4c32b2e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:13.925950 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311abb6-18dc-47bf-9d3b-95d4c32b2e9f None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:13.926227 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311abb6-18dc-47bf-9d3b-95d4c32b2e9f None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:13.926501 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311abb6-18dc-47bf-9d3b-95d4c32b2e9f None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:13.927117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311abb6-18dc-47bf-9d3b-95d4c32b2e9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:13.927227 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42f693-1c8f-4f20-b247-11ed13d4cfa2 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:13.928613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311abb6-18dc-47bf-9d3b-95d4c32b2e9f None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:13.935977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 351/704] 10.222.0.36 () {56 vars in 1087 bytes} [Sun Oct 30 00:36:13 2022] GET /identity/v3/services => generated 1497 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:36:13.964533 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:13.985890 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607252a-8f99-484e-950c-2f8b7bef3fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:13.986874 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607252a-8f99-484e-950c-2f8b7bef3fd5 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:13.987297 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607252a-8f99-484e-950c-2f8b7bef3fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:13.987737 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607252a-8f99-484e-950c-2f8b7bef3fd5 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:13.988601 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607252a-8f99-484e-950c-2f8b7bef3fd5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:13.989063 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607252a-8f99-484e-950c-2f8b7bef3fd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:13.990632 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607252a-8f99-484e-950c-2f8b7bef3fd5 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:13.998185 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 354/705] 10.222.0.36 () {56 vars in 1126 bytes} [Sun Oct 30 00:36:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:14.008112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311abb6-18dc-47bf-9d3b-95d4c32b2e9f None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:14.124521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311abb6-18dc-47bf-9d3b-95d4c32b2e9f None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:14.163642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2a124-1470-4c69-a40b-c8f40d7802e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:14.164298 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2a124-1470-4c69-a40b-c8f40d7802e6 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:14.164637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2a124-1470-4c69-a40b-c8f40d7802e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:14.164915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2a124-1470-4c69-a40b-c8f40d7802e6 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:14.165553 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2a124-1470-4c69-a40b-c8f40d7802e6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:14.165903 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2a124-1470-4c69-a40b-c8f40d7802e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:14.167290 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2a124-1470-4c69-a40b-c8f40d7802e6 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:14.174021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 352/706] 10.222.0.36 () {56 vars in 1126 bytes} [Sun Oct 30 00:36:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:14.182795 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607252a-8f99-484e-950c-2f8b7bef3fd5 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:14.318085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607252a-8f99-484e-950c-2f8b7bef3fd5 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:14.365060 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6837c-d444-46e5-a0f2-bf4936919682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:14.366051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6837c-d444-46e5-a0f2-bf4936919682 None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:14.366516 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6837c-d444-46e5-a0f2-bf4936919682 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:14.366989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6837c-d444-46e5-a0f2-bf4936919682 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:14.367885 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6837c-d444-46e5-a0f2-bf4936919682 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:14.369558 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6837c-d444-46e5-a0f2-bf4936919682 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:14.396964 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 355/707] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:36:14 2022] POST /identity/v3/endpoints => generated 347 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:36:14.637862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ba313-f543-47c4-bc99-20b5ae1ce31c None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:14.638105 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ba313-f543-47c4-bc99-20b5ae1ce31c None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:14.638516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ba313-f543-47c4-bc99-20b5ae1ce31c None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:14.640162 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 353/708] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:14.645800 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a4e54-5e21-4b99-a7d7-6222d5dd2171 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:14.646092 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a4e54-5e21-4b99-a7d7-6222d5dd2171 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:14.646364 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a4e54-5e21-4b99-a7d7-6222d5dd2171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:14.671284 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-023a4e54-5e21-4b99-a7d7-6222d5dd2171 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:14.672531 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-023a4e54-5e21-4b99-a7d7-6222d5dd2171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:14.711839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:14.757709 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 356/709] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:15.025951 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ba313-f543-47c4-bc99-20b5ae1ce31c None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:15.040948 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:15.167878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ba313-f543-47c4-bc99-20b5ae1ce31c None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:15.207791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec6df2-2190-4110-848a-4972dc0806a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:15.208487 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec6df2-2190-4110-848a-4972dc0806a0 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:15.208770 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec6df2-2190-4110-848a-4972dc0806a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:15.209045 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec6df2-2190-4110-848a-4972dc0806a0 None admin] PATH_INFO: `/v3/services/image` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:15.209638 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec6df2-2190-4110-848a-4972dc0806a0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:15.211031 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec6df2-2190-4110-848a-4972dc0806a0 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:15.221283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-7bec6df2-2190-4110-848a-4972dc0806a0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 30 00:36:15.221283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:15.221283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:15.221283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:15.221283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:15.221283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:15.221283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:15.221283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:15.221283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 30 00:36:15.221283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:15.221283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:15.221283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:15.221283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:15.221283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:15.221283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:15.221283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:15.221283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:15.222355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:15.222355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:15.222355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:15.222355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:15.222355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:15.222355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:15.222355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:15.222355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:15.222355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:15.222355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:15.222355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:15.222355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:15.222355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:15.222355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:15.222355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:15.222355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:15.223211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:15.223211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:15.223211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:15.223211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:15.223211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:15.223211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:15.223211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:15.223211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:15.223211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:15.223211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:15.223211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:15.223211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:15.223211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 30 00:36:15.223211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:15.223211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 354/710] 10.222.0.36 () {56 vars in 1105 bytes} [Sun Oct 30 00:36:15 2022] GET /identity/v3/services/image => generated 86 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:15.228470 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a4e54-5e21-4b99-a7d7-6222d5dd2171 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:15.250392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764e705-a396-4510-b561-61dfc0b0c00c None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:15.250983 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764e705-a396-4510-b561-61dfc0b0c00c None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:15.251407 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764e705-a396-4510-b561-61dfc0b0c00c None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:15.253270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 355/711] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:36:15.258966 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714467b-00d8-495f-8310-82cbcf5a8609 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:15.259493 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714467b-00d8-495f-8310-82cbcf5a8609 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:15.259916 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714467b-00d8-495f-8310-82cbcf5a8609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:15.282733 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-0714467b-00d8-495f-8310-82cbcf5a8609 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:15.284110 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-0714467b-00d8-495f-8310-82cbcf5a8609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:15.352994 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a4e54-5e21-4b99-a7d7-6222d5dd2171 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:15.392108 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 356/712] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:36:15.404668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e3be9-947b-401c-b5fc-f234ccd81e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:15.405621 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e3be9-947b-401c-b5fc-f234ccd81e25 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:15.405907 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e3be9-947b-401c-b5fc-f234ccd81e25 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:15.406253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e3be9-947b-401c-b5fc-f234ccd81e25 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:15.406907 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e3be9-947b-401c-b5fc-f234ccd81e25 None admin] RBAC: Adding query filter params (name=image) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:15.407212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e3be9-947b-401c-b5fc-f234ccd81e25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:15.408659 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e3be9-947b-401c-b5fc-f234ccd81e25 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:15.414198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 357/713] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:15.421180 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714467b-00d8-495f-8310-82cbcf5a8609 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:15.436761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:15.531555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714467b-00d8-495f-8310-82cbcf5a8609 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:15.569063 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260c926-765f-4101-8235-c5370be2e421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:15.569601 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260c926-765f-4101-8235-c5370be2e421 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:15.569882 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260c926-765f-4101-8235-c5370be2e421 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:15.570161 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260c926-765f-4101-8235-c5370be2e421 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:15.570802 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260c926-765f-4101-8235-c5370be2e421 None admin] RBAC: Adding query filter params (type=image) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:15.571102 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260c926-765f-4101-8235-c5370be2e421 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:15.572519 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260c926-765f-4101-8235-c5370be2e421 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:15.578798 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 357/714] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:15 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:15.676964 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e3be9-947b-401c-b5fc-f234ccd81e25 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:15.718325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:15.811400 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e3be9-947b-401c-b5fc-f234ccd81e25 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:15.852603 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afa1e9-5682-41ad-aeeb-d3063b96e23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:15.853170 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afa1e9-5682-41ad-aeeb-d3063b96e23e None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:15.853659 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afa1e9-5682-41ad-aeeb-d3063b96e23e None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:15.853942 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afa1e9-5682-41ad-aeeb-d3063b96e23e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:15.854538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9afa1e9-5682-41ad-aeeb-d3063b96e23e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:15.856564 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9afa1e9-5682-41ad-aeeb-d3063b96e23e None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:15.866468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-f9afa1e9-5682-41ad-aeeb-d3063b96e23e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 30 00:36:15.866468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:15.866468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:15.866468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:15.866468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:15.866468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:15.866468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:15.866468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:15.866468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 30 00:36:15.866468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:15.866468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:15.866468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:15.866468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:15.866468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:15.866468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:15.866468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:15.866468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:15.867518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:15.867518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:15.867518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:15.867518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:15.867518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:15.867518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:15.867518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:15.867518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:15.867518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:15.867518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:15.867518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:15.867518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:15.867518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:15.867518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:15.867518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:15.867518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:15.868484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:15.868484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:15.868484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:15.868484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:15.868484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:15.868484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:15.868484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:15.868484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:15.868484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:15.868484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:15.868484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:15.868484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:15.868484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 30 00:36:15.868484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:15.868484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 358/715] 10.222.0.36 () {56 vars in 1114 bytes} [Sun Oct 30 00:36:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:15.874373 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260c926-765f-4101-8235-c5370be2e421 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:15.954470 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:15.991977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260c926-765f-4101-8235-c5370be2e421 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:16.031060 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1320a-04db-4234-a861-04033f0b14ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:16.031817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1320a-04db-4234-a861-04033f0b14ab None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:16.031974 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1320a-04db-4234-a861-04033f0b14ab None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:16.032239 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1320a-04db-4234-a861-04033f0b14ab None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:16.032947 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1320a-04db-4234-a861-04033f0b14ab None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:16.033250 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1320a-04db-4234-a861-04033f0b14ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:16.034619 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1320a-04db-4234-a861-04033f0b14ab None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:16.040128 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 358/716] 10.222.0.36 () {56 vars in 1114 bytes} [Sun Oct 30 00:36:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:16.046539 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afa1e9-5682-41ad-aeeb-d3063b96e23e None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:16.158329 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:16.165788 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afa1e9-5682-41ad-aeeb-d3063b96e23e None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:16.210583 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f561e-8c15-4538-b4be-efc9102d73cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:16.211213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f561e-8c15-4538-b4be-efc9102d73cc None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:16.211495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f561e-8c15-4538-b4be-efc9102d73cc None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:16.211771 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f561e-8c15-4538-b4be-efc9102d73cc None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:16.212446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f561e-8c15-4538-b4be-efc9102d73cc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:16.212748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f561e-8c15-4538-b4be-efc9102d73cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:16.214135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f561e-8c15-4538-b4be-efc9102d73cc None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:16.221249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 359/717] 10.222.0.36 () {56 vars in 1114 bytes} [Sun Oct 30 00:36:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:16.226997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1320a-04db-4234-a861-04033f0b14ab None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:16.356649 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1320a-04db-4234-a861-04033f0b14ab None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:16.393178 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184344d7-7683-4187-8c92-f7c72c9739fa None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:16.393534 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184344d7-7683-4187-8c92-f7c72c9739fa None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:16.393735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184344d7-7683-4187-8c92-f7c72c9739fa None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:16.395516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:16.396217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 360/718] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:36:16.401320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03886a7d-296f-413f-8f0a-34ef0641cdc4 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:16.401491 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8738e9d-5f82-471b-8c97-ddc379243ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:16.401589 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03886a7d-296f-413f-8f0a-34ef0641cdc4 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:16.401973 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03886a7d-296f-413f-8f0a-34ef0641cdc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:16.402089 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8738e9d-5f82-471b-8c97-ddc379243ba6 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:16.402305 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8738e9d-5f82-471b-8c97-ddc379243ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:16.403068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8738e9d-5f82-471b-8c97-ddc379243ba6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:16.403758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8738e9d-5f82-471b-8c97-ddc379243ba6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:16.407413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8738e9d-5f82-471b-8c97-ddc379243ba6 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:16.427951 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-03886a7d-296f-413f-8f0a-34ef0641cdc4 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:16.429169 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-03886a7d-296f-413f-8f0a-34ef0641cdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:16.435384 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 359/719] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:36:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:36:16.528780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 361/720] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:36:16.804603 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8738e9d-5f82-471b-8c97-ddc379243ba6 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:16.921468 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8738e9d-5f82-471b-8c97-ddc379243ba6 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:16.932691 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19ab69-9762-42e6-9623-00a372798624 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:16.933470 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19ab69-9762-42e6-9623-00a372798624 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:16.933998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19ab69-9762-42e6-9623-00a372798624 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:16.936043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 362/721] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:16.941901 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92812c8-7e9b-4057-a1fb-e0392d57c500 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:16.942514 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92812c8-7e9b-4057-a1fb-e0392d57c500 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:16.943009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92812c8-7e9b-4057-a1fb-e0392d57c500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:16.962101 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccf7a9-49e6-4671-bfdd-9ced0bfef17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:16.962762 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccf7a9-49e6-4671-bfdd-9ced0bfef17f None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:16.963042 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccf7a9-49e6-4671-bfdd-9ced0bfef17f None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:16.963318 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccf7a9-49e6-4671-bfdd-9ced0bfef17f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:16.963909 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caccf7a9-49e6-4671-bfdd-9ced0bfef17f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:16.964877 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-a92812c8-7e9b-4057-a1fb-e0392d57c500 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:16.965986 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caccf7a9-49e6-4671-bfdd-9ced0bfef17f None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:16.967481 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a92812c8-7e9b-4057-a1fb-e0392d57c500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:16.975805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-caccf7a9-49e6-4671-bfdd-9ced0bfef17f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:36:16.975805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:16.975805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:16.975805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:16.975805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:16.975805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:16.975805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:16.975805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:16.975805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:36:16.975805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:16.975805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:16.975805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:16.975805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:16.975805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:16.975805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:16.975805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:16.975805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:16.976885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:16.976885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:16.976885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:16.976885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:16.976885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:16.976885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:16.976885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:16.976885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:16.976885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:16.976885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:16.976885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:16.976885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:16.976885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:16.976885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:16.976885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:16.976885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:16.977716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:16.977716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:16.977716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:16.977716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:16.977716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:16.977716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:16.977716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:16.977716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:16.977716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:16.977716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:16.977716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:16.977716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:16.977716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:36:16.977716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:16.982666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 360/722] 10.222.0.36 () {56 vars in 1126 bytes} [Sun Oct 30 00:36:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:16.984056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccf7a9-49e6-4671-bfdd-9ced0bfef17f None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:17.051278 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:17.056231 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 363/723] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:36:17.116827 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccf7a9-49e6-4671-bfdd-9ced0bfef17f None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:17.162020 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76102604-ffe2-4df9-a41d-57b2099c34e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:17.162638 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76102604-ffe2-4df9-a41d-57b2099c34e5 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:17.162915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76102604-ffe2-4df9-a41d-57b2099c34e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:17.163185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76102604-ffe2-4df9-a41d-57b2099c34e5 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:17.163803 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76102604-ffe2-4df9-a41d-57b2099c34e5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:17.164122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76102604-ffe2-4df9-a41d-57b2099c34e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:17.165527 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76102604-ffe2-4df9-a41d-57b2099c34e5 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:17.172484 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 361/724] 10.222.0.36 () {56 vars in 1122 bytes} [Sun Oct 30 00:36:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:17.181512 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92812c8-7e9b-4057-a1fb-e0392d57c500 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:17.199140 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:17.295867 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92812c8-7e9b-4057-a1fb-e0392d57c500 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:17.336395 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a510637-223a-4fe4-af81-77c6e51816bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:17.337354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a510637-223a-4fe4-af81-77c6e51816bf None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:17.337813 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a510637-223a-4fe4-af81-77c6e51816bf None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:17.338229 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a510637-223a-4fe4-af81-77c6e51816bf None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:17.339267 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a510637-223a-4fe4-af81-77c6e51816bf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:17.339780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a510637-223a-4fe4-af81-77c6e51816bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:17.341896 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a510637-223a-4fe4-af81-77c6e51816bf None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:17.355444 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 364/725] 10.222.0.36 () {56 vars in 1122 bytes} [Sun Oct 30 00:36:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:17.358068 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a510637-223a-4fe4-af81-77c6e51816bf None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:17.358201 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76102604-ffe2-4df9-a41d-57b2099c34e5 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:17.463989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:17.494790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a510637-223a-4fe4-af81-77c6e51816bf None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:17.497318 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76102604-ffe2-4df9-a41d-57b2099c34e5 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:17.525395 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:17.549758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042024ac-37ee-4737-9115-d0648aa2e2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:17.550602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042024ac-37ee-4737-9115-d0648aa2e2f9 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:17.550962 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042024ac-37ee-4737-9115-d0648aa2e2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:17.551390 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042024ac-37ee-4737-9115-d0648aa2e2f9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:17.552026 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042024ac-37ee-4737-9115-d0648aa2e2f9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:17.553097 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f9ba3-1d50-4569-90e4-321c27edf012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:17.553633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042024ac-37ee-4737-9115-d0648aa2e2f9 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:17.554236 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f9ba3-1d50-4569-90e4-321c27edf012 None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:17.554754 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f9ba3-1d50-4569-90e4-321c27edf012 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:17.555200 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f9ba3-1d50-4569-90e4-321c27edf012 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:17.556516 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f9ba3-1d50-4569-90e4-321c27edf012 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:17.557930 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f9ba3-1d50-4569-90e4-321c27edf012 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:17.570425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-042024ac-37ee-4737-9115-d0648aa2e2f9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 30 00:36:17.570425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:17.570425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:17.570425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:17.570425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:17.570425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:17.570425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:17.570425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:17.570425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 30 00:36:17.570425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:17.570425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:17.570425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:17.570425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:17.570425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:17.570425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:17.570425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:17.570425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:17.571758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:17.571758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:17.571758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:17.571758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:17.571758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:17.571758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:17.571758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:17.571758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:17.571758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:17.571758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:17.571758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:17.571758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:17.571758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:17.571758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:17.571758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:17.571758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:17.572687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:17.572687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:17.572687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:17.572687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:17.572687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:17.572687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:17.572687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:17.572687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:17.572687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:17.572687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:17.572687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:17.572687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:17.572687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 30 00:36:17.572687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:17.576284 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 362/726] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:36:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:17.578524 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042024ac-37ee-4737-9115-d0648aa2e2f9 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:17.592784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 365/727] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:36:17 2022] POST /identity/v3/endpoints => generated 349 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:36:17.707607 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042024ac-37ee-4737-9115-d0648aa2e2f9 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:17.756244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca01128-7fe9-46fa-a4e3-e8ee22962726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:17.757287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca01128-7fe9-46fa-a4e3-e8ee22962726 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:17.757764 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca01128-7fe9-46fa-a4e3-e8ee22962726 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:17.758232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca01128-7fe9-46fa-a4e3-e8ee22962726 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:17.759070 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca01128-7fe9-46fa-a4e3-e8ee22962726 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:17.759571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca01128-7fe9-46fa-a4e3-e8ee22962726 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:17.761448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca01128-7fe9-46fa-a4e3-e8ee22962726 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:17.768313 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 363/728] 10.222.0.36 () {56 vars in 1112 bytes} [Sun Oct 30 00:36:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:17.775492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f9ba3-1d50-4569-90e4-321c27edf012 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:17.904966 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f9ba3-1d50-4569-90e4-321c27edf012 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:17.918820 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719924c-f363-4705-ac13-d059f911a57a None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:17.919175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719924c-f363-4705-ac13-d059f911a57a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:17.919382 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719924c-f363-4705-ac13-d059f911a57a None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:17.921029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 364/729] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:17.926532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04367538-6b77-48a3-bccf-6006776c410e None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:17.926694 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04367538-6b77-48a3-bccf-6006776c410e None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:17.926956 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04367538-6b77-48a3-bccf-6006776c410e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:17.947043 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-04367538-6b77-48a3-bccf-6006776c410e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:17.948287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-04367538-6b77-48a3-bccf-6006776c410e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:17.948746 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce8481-569e-4735-ac8f-0f5bf1d647e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:17.949656 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce8481-569e-4735-ac8f-0f5bf1d647e9 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:17.950159 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce8481-569e-4735-ac8f-0f5bf1d647e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:17.950935 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce8481-569e-4735-ac8f-0f5bf1d647e9 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:17.952783 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce8481-569e-4735-ac8f-0f5bf1d647e9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:17.953758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce8481-569e-4735-ac8f-0f5bf1d647e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:17.954620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce8481-569e-4735-ac8f-0f5bf1d647e9 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:17.956297 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:17.966957 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 366/730] 10.222.0.36 () {56 vars in 1112 bytes} [Sun Oct 30 00:36:17 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:18.002151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:18.039985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 365/731] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:18.313200 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce8481-569e-4735-ac8f-0f5bf1d647e9 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:18.434671 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce8481-569e-4735-ac8f-0f5bf1d647e9 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:18.473735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf458cd7-6cc1-4067-b90d-ec1278004227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:18.475165 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf458cd7-6cc1-4067-b90d-ec1278004227 None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:18.475641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf458cd7-6cc1-4067-b90d-ec1278004227 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:18.476151 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf458cd7-6cc1-4067-b90d-ec1278004227 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:18.477013 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf458cd7-6cc1-4067-b90d-ec1278004227 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:18.478617 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf458cd7-6cc1-4067-b90d-ec1278004227 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:18.497287 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 367/732] 10.222.0.36 () {60 vars in 1141 bytes} [Sun Oct 30 00:36:18 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:36:18.967949 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14511ce-d5a1-442e-8f60-ef4962849aea None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:18.968496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14511ce-d5a1-442e-8f60-ef4962849aea None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:18.968496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14511ce-d5a1-442e-8f60-ef4962849aea None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:18.970129 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 366/733] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:18.975727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab254d65-54ec-4ec8-b2c9-5815f1c21b6a None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:18.976014 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab254d65-54ec-4ec8-b2c9-5815f1c21b6a None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:18.976324 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab254d65-54ec-4ec8-b2c9-5815f1c21b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:18.996605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-ab254d65-54ec-4ec8-b2c9-5815f1c21b6a None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:18.997836 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-ab254d65-54ec-4ec8-b2c9-5815f1c21b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:19.060364 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:19.087480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 368/734] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:19.376064 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14511ce-d5a1-442e-8f60-ef4962849aea None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:19.503096 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14511ce-d5a1-442e-8f60-ef4962849aea None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:19.544685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0284a6e-3bb9-4c78-b79a-9973e440f1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:19.545356 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0284a6e-3bb9-4c78-b79a-9973e440f1f1 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:19.545635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0284a6e-3bb9-4c78-b79a-9973e440f1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:19.545904 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0284a6e-3bb9-4c78-b79a-9973e440f1f1 None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:19.546553 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0284a6e-3bb9-4c78-b79a-9973e440f1f1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:19.547980 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0284a6e-3bb9-4c78-b79a-9973e440f1f1 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:19.550807 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: WARNING keystone.server.flask.common [None req-f0284a6e-3bb9-4c78-b79a-9973e440f1f1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 30 00:36:19.570856 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.sql.core [None req-f0284a6e-3bb9-4c78-b79a-9973e440f1f1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 30 00:36:19.570856 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 30 00:36:19.570856 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [parameters: {'id': '4a952aceb9db4338b4297fce750bdb3f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 30 00:36:19.570856 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88201) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 30 00:36:19.580155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-f0284a6e-3bb9-4c78-b79a-9973e440f1f1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4a952aceb9db4338b4297fce750bdb3f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4a952aceb9db4338b4297fce750bdb3f. Oct 30 00:36:19.580155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:19.580155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:36:19.580155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:36:19.580155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:36:19.580155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:36:19.580155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:36:19.580155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:36:19.580155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:36:19.580155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:36:19.580155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:36:19.580155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:36:19.580155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:36:19.581022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application result.read() Oct 30 00:36:19.581022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:36:19.581022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:36:19.581022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:36:19.581022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:36:19.581022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:36:19.581022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:36:19.581022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:36:19.581022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:36:19.581022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 30 00:36:19.581022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:19.581022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 30 00:36:19.581022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:19.581022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:19.581022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 30 00:36:19.581022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 30 00:36:19.581022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 30 00:36:19.581022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 30 00:36:19.581994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:36:19.581994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:36:19.581994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 30 00:36:19.581994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application yield resource Oct 30 00:36:19.581994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:36:19.581994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:36:19.581994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 30 00:36:19.581994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.session.rollback() Oct 30 00:36:19.581994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 30 00:36:19.581994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.force_reraise() Oct 30 00:36:19.581994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 30 00:36:19.581994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise self.value Oct 30 00:36:19.581994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 30 00:36:19.581994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 30 00:36:19.581994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 30 00:36:19.581994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application session.commit() Oct 30 00:36:19.581994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 30 00:36:19.583066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 30 00:36:19.583066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 30 00:36:19.583066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._prepare_impl() Oct 30 00:36:19.583066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 30 00:36:19.583066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.session.flush() Oct 30 00:36:19.583066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 30 00:36:19.583066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._flush(objects) Oct 30 00:36:19.583066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 30 00:36:19.583066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 30 00:36:19.583066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 30 00:36:19.583066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application compat.raise_( Oct 30 00:36:19.583066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:36:19.583066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception Oct 30 00:36:19.583066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 30 00:36:19.583066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application flush_context.execute() Oct 30 00:36:19.583066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 30 00:36:19.583066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rec.execute(self) Oct 30 00:36:19.584011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 30 00:36:19.584011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 30 00:36:19.584011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 30 00:36:19.584011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 30 00:36:19.584011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 30 00:36:19.584011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 30 00:36:19.584011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 30 00:36:19.584011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 30 00:36:19.584011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 30 00:36:19.584011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 30 00:36:19.584011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 30 00:36:19.584011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 30 00:36:19.584011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 30 00:36:19.584011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 30 00:36:19.584011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 30 00:36:19.584880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 30 00:36:19.584880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:36:19.584880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception Oct 30 00:36:19.584880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:36:19.584880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:36:19.584880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:36:19.584880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:36:19.584880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:36:19.584880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:36:19.584880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:36:19.584880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:36:19.584880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:36:19.584880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:36:19.584880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:36:19.584880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application result.read() Oct 30 00:36:19.584880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:36:19.584880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:36:19.585781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:36:19.585781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:36:19.585781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:36:19.585781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:36:19.585781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:36:19.585781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:36:19.585781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 30 00:36:19.585781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 30 00:36:19.585781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [parameters: {'id': '4a952aceb9db4338b4297fce750bdb3f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 30 00:36:19.585781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 30 00:36:19.585781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:19.585781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:19.585781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:19.585781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:19.585781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 30 00:36:19.588255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 30 00:36:19.588255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 30 00:36:19.588255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 30 00:36:19.588255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 30 00:36:19.588255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:19.588255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:19.588255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:19.588255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:19.588255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:19.588255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:19.588255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:19.588255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:19.588255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:19.588255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:19.588255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:19.588255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:19.588255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:19.590088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:19.590088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 30 00:36:19.590088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 30 00:36:19.590088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:19.590088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:19.590088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 30 00:36:19.590088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 30 00:36:19.590088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4a952aceb9db4338b4297fce750bdb3f. Oct 30 00:36:19.590088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:19.590088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 367/735] 10.222.0.36 () {60 vars in 1140 bytes} [Sun Oct 30 00:36:19 2022] POST /identity/v3/projects => generated 262 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:19.591334 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab254d65-54ec-4ec8-b2c9-5815f1c21b6a None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:19.698839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab254d65-54ec-4ec8-b2c9-5815f1c21b6a None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:19.751712 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8548306-2071-4337-ba2b-8ce08343557a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:19.752392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8548306-2071-4337-ba2b-8ce08343557a None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:19.752671 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8548306-2071-4337-ba2b-8ce08343557a None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:19.752981 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8548306-2071-4337-ba2b-8ce08343557a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:19.761839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8548306-2071-4337-ba2b-8ce08343557a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:19.764151 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8548306-2071-4337-ba2b-8ce08343557a None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:19.777203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-a8548306-2071-4337-ba2b-8ce08343557a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:36:19.777203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:19.777203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:19.777203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:19.777203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:19.777203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:19.777203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:19.777203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:19.777203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:19.777203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:19.777203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:19.777203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:19.777203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:36:19.777203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:36:19.777203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:36:19.779034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:36:19.779034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:19.779034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:19.779034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:19.779034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:19.779034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:19.779034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:19.779034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:19.779034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:19.779034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:19.779034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:19.779034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:19.779034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:19.779034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:19.779034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:19.779034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:19.779034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:19.780094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:36:19.780094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:36:19.780094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:36:19.780094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:36:19.780094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:36:19.780094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:36:19.780094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:36:19.780094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:19.780094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 369/736] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:36:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:19.787643 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0284a6e-3bb9-4c78-b79a-9973e440f1f1 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:19.834904 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:19.921348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd46a7-9128-49dd-9e5e-a84ab08e070f None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:19.921680 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd46a7-9128-49dd-9e5e-a84ab08e070f None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:19.921894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd46a7-9128-49dd-9e5e-a84ab08e070f None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:19.923501 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 370/737] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:36:19.923755 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0284a6e-3bb9-4c78-b79a-9973e440f1f1 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:19.929524 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222936a8-e2d1-4961-944f-54180028e32c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:19.930080 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222936a8-e2d1-4961-944f-54180028e32c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:19.930549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222936a8-e2d1-4961-944f-54180028e32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:19.952153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-222936a8-e2d1-4961-944f-54180028e32c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:19.953703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-222936a8-e2d1-4961-944f-54180028e32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:19.961198 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19bed60-4455-4b90-8012-5963517953ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:19.961816 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bed60-4455-4b90-8012-5963517953ce None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:19.962093 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bed60-4455-4b90-8012-5963517953ce None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:19.962393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bed60-4455-4b90-8012-5963517953ce None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:19.963140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19bed60-4455-4b90-8012-5963517953ce None admin] RBAC: Adding query filter params (name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:19.963447 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19bed60-4455-4b90-8012-5963517953ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:19.965382 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19bed60-4455-4b90-8012-5963517953ce None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:19.990465 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 368/738] 10.222.0.36 () {56 vars in 1112 bytes} [Sun Oct 30 00:36:19 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:20.003597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:20.036988 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 371/739] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:20.332749 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19bed60-4455-4b90-8012-5963517953ce None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:20.380131 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987e9e7-ef45-47d8-a5ca-e679608fb13e None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:20.380131 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987e9e7-ef45-47d8-a5ca-e679608fb13e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:20.380657 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987e9e7-ef45-47d8-a5ca-e679608fb13e None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:20.382517 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 372/740] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:20.389216 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf859e5-6e46-4ae8-a9b3-c177abffe359 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:20.389838 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf859e5-6e46-4ae8-a9b3-c177abffe359 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:20.390314 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf859e5-6e46-4ae8-a9b3-c177abffe359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:20.410316 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-5cf859e5-6e46-4ae8-a9b3-c177abffe359 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:20.412008 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-5cf859e5-6e46-4ae8-a9b3-c177abffe359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:20.432984 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:20.449324 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19bed60-4455-4b90-8012-5963517953ce None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:20.487965 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d288c-e013-4293-ba78-b3aefd44c29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:20.488702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d288c-e013-4293-ba78-b3aefd44c29b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:20.488982 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d288c-e013-4293-ba78-b3aefd44c29b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:20.489303 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d288c-e013-4293-ba78-b3aefd44c29b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:20.489895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d288c-e013-4293-ba78-b3aefd44c29b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:20.491354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d288c-e013-4293-ba78-b3aefd44c29b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:20.501867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-af9d288c-e013-4293-ba78-b3aefd44c29b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:36:20.501867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:20.501867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:20.501867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:20.501867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:20.501867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:20.501867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:20.501867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:20.501867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:36:20.501867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:20.501867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:20.501867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:20.501867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:20.501867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:20.501867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:20.501867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:20.501867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:20.503052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:20.503052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:20.503052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:20.503052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:20.503052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:20.503052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:20.503052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:20.503052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:20.503052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:20.503052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:20.503052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:20.503052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:20.503052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:20.503052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:20.503052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:20.503052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:20.503908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:20.503908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:20.503908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:20.503908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:20.503908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:20.503908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:20.503908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:20.503908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:20.503908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:20.503908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:20.503908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:20.503908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:20.503908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:36:20.503908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:20.503908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 369/741] 10.222.0.36 () {56 vars in 1126 bytes} [Sun Oct 30 00:36:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:20.512138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d288c-e013-4293-ba78-b3aefd44c29b None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:20.518384 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 373/742] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:36:20.629911 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d288c-e013-4293-ba78-b3aefd44c29b None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:20.666516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f2d92-9762-4a96-85d0-55ce1d3d4b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:20.667142 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f2d92-9762-4a96-85d0-55ce1d3d4b67 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:20.667425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f2d92-9762-4a96-85d0-55ce1d3d4b67 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:20.667703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f2d92-9762-4a96-85d0-55ce1d3d4b67 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:20.669232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f2d92-9762-4a96-85d0-55ce1d3d4b67 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:20.669927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f2d92-9762-4a96-85d0-55ce1d3d4b67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:20.671323 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f2d92-9762-4a96-85d0-55ce1d3d4b67 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:20.677078 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 370/743] 10.222.0.36 () {56 vars in 1122 bytes} [Sun Oct 30 00:36:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:20.682810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf859e5-6e46-4ae8-a9b3-c177abffe359 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:20.792729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf859e5-6e46-4ae8-a9b3-c177abffe359 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:20.796202 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:20.807496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f2d92-9762-4a96-85d0-55ce1d3d4b67 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:20.829130 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:20.853753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be2882-82d9-4172-bfc6-baad7f5d1633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:20.854769 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be2882-82d9-4172-bfc6-baad7f5d1633 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:20.855253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be2882-82d9-4172-bfc6-baad7f5d1633 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:20.855755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be2882-82d9-4172-bfc6-baad7f5d1633 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:20.857487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be2882-82d9-4172-bfc6-baad7f5d1633 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:20.858041 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be2882-82d9-4172-bfc6-baad7f5d1633 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:20.859647 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be2882-82d9-4172-bfc6-baad7f5d1633 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:20.865770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 374/744] 10.222.0.36 () {56 vars in 1122 bytes} [Sun Oct 30 00:36:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:20.874982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be2882-82d9-4172-bfc6-baad7f5d1633 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:20.955982 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f2d92-9762-4a96-85d0-55ce1d3d4b67 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:20.995948 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d9a82-a4a1-47fc-ab69-17eeafaf1657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:20.996664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d9a82-a4a1-47fc-ab69-17eeafaf1657 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:20.996932 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d9a82-a4a1-47fc-ab69-17eeafaf1657 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:20.998680 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d9a82-a4a1-47fc-ab69-17eeafaf1657 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:20.999683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d9a82-a4a1-47fc-ab69-17eeafaf1657 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:21.002730 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d9a82-a4a1-47fc-ab69-17eeafaf1657 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:21.016661 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be2882-82d9-4172-bfc6-baad7f5d1633 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:21.022773 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 371/745] 10.222.0.36 () {60 vars in 1141 bytes} [Sun Oct 30 00:36:20 2022] POST /identity/v3/services => generated 242 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:36:21.024765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d92abd-e8dc-48c3-86e9-e8085e322d1a None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:21.025251 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d92abd-e8dc-48c3-86e9-e8085e322d1a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:21.025544 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d92abd-e8dc-48c3-86e9-e8085e322d1a None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:21.027079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 372/746] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:21.036135 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead7a94-edce-4787-9ecf-bd29e074dcfe None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:21.036442 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead7a94-edce-4787-9ecf-bd29e074dcfe None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:21.036774 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead7a94-edce-4787-9ecf-bd29e074dcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:21.072847 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-dead7a94-edce-4787-9ecf-bd29e074dcfe None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:21.077431 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-dead7a94-edce-4787-9ecf-bd29e074dcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:21.095335 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2d5a8-222f-49b3-89f0-ef652920dc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:21.095972 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2d5a8-222f-49b3-89f0-ef652920dc37 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:21.096287 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2d5a8-222f-49b3-89f0-ef652920dc37 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:21.096566 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2d5a8-222f-49b3-89f0-ef652920dc37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:21.097930 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2d5a8-222f-49b3-89f0-ef652920dc37 None admin] RBAC: Adding query filter params (interface=admin, service_id=3098d10f6fd8491b8a261bf0827e32ea, region_id=RegionOne) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:21.098259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2d5a8-222f-49b3-89f0-ef652920dc37 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:21.100041 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2d5a8-222f-49b3-89f0-ef652920dc37 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:21.116804 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 375/747] 10.222.0.36 () {56 vars in 1249 bytes} [Sun Oct 30 00:36:20 2022] GET /identity/v3/endpoints?service_id=3098d10f6fd8491b8a261bf0827e32ea&interface=admin®ion_id=RegionOne => generated 193 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:21.118278 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2d5a8-222f-49b3-89f0-ef652920dc37 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:21.162911 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:21.188113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 373/748] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:21 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:36:21.257050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2d5a8-222f-49b3-89f0-ef652920dc37 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:21.305106 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2a9d9-5c54-4296-826e-2733c490d622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:21.306080 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2a9d9-5c54-4296-826e-2733c490d622 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:21.306541 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2a9d9-5c54-4296-826e-2733c490d622 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:21.306986 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2a9d9-5c54-4296-826e-2733c490d622 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:21.307914 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2a9d9-5c54-4296-826e-2733c490d622 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:21.311859 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2a9d9-5c54-4296-826e-2733c490d622 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:21.320459 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 376/749] 10.222.0.36 () {56 vars in 1087 bytes} [Sun Oct 30 00:36:21 2022] GET /identity/v3/services => generated 1957 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:36:21.492328 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead7a94-edce-4787-9ecf-bd29e074dcfe None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:21.495826 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:21.617052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead7a94-edce-4787-9ecf-bd29e074dcfe None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:21.655843 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23df82f-4c7c-44ff-a8a7-f57889dac6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:21.656951 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23df82f-4c7c-44ff-a8a7-f57889dac6dc None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:21.657431 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23df82f-4c7c-44ff-a8a7-f57889dac6dc None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:21.657896 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23df82f-4c7c-44ff-a8a7-f57889dac6dc None admin] PATH_INFO: `/v3/services/image` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:21.658861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23df82f-4c7c-44ff-a8a7-f57889dac6dc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:21.660486 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23df82f-4c7c-44ff-a8a7-f57889dac6dc None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:21.674083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-d23df82f-4c7c-44ff-a8a7-f57889dac6dc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 30 00:36:21.674083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:21.674083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:21.674083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:21.674083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:21.674083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:21.674083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:21.674083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:21.674083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 30 00:36:21.674083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:21.674083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:21.674083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:21.674083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:21.674083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:21.674083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:21.674083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:21.674083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:21.675160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:21.675160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:21.675160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:21.675160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:21.675160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:21.675160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:21.675160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:21.675160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:21.675160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:21.675160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:21.675160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:21.675160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:21.675160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:21.675160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:21.675160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:21.675160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:21.675994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:21.675994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:21.675994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:21.675994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:21.675994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:21.675994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:21.675994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:21.675994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:21.675994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:21.675994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:21.675994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:21.675994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:21.675994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 30 00:36:21.675994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:21.684274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 374/750] 10.222.0.36 () {56 vars in 1105 bytes} [Sun Oct 30 00:36:21 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:21.688321 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2a9d9-5c54-4296-826e-2733c490d622 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:21.802498 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:21.812885 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2a9d9-5c54-4296-826e-2733c490d622 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:21.851848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ff62a-25f9-4197-9680-3159f277bc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:21.852501 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ff62a-25f9-4197-9680-3159f277bc3a None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:21.852775 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ff62a-25f9-4197-9680-3159f277bc3a None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:21.853044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ff62a-25f9-4197-9680-3159f277bc3a None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:21.853729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ff62a-25f9-4197-9680-3159f277bc3a None admin] RBAC: Adding query filter params (name=image) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:21.854028 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ff62a-25f9-4197-9680-3159f277bc3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:21.855469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ff62a-25f9-4197-9680-3159f277bc3a None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:21.871261 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 377/751] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:21.877125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23df82f-4c7c-44ff-a8a7-f57889dac6dc None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:21.952725 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:21.991165 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23df82f-4c7c-44ff-a8a7-f57889dac6dc None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:22.033875 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722adefb-448a-4595-be4b-5e30c8c071eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:22.034549 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722adefb-448a-4595-be4b-5e30c8c071eb None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:22.034797 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722adefb-448a-4595-be4b-5e30c8c071eb None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:22.035029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722adefb-448a-4595-be4b-5e30c8c071eb None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:22.035664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722adefb-448a-4595-be4b-5e30c8c071eb None admin] RBAC: Adding query filter params (type=image) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:22.035962 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722adefb-448a-4595-be4b-5e30c8c071eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:22.037369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722adefb-448a-4595-be4b-5e30c8c071eb None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:22.046189 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 375/752] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:22.051429 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ff62a-25f9-4197-9680-3159f277bc3a None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:22.174424 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ff62a-25f9-4197-9680-3159f277bc3a None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:22.228464 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d67ec-8af3-471c-b1c7-5dc293802878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:22.229074 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d67ec-8af3-471c-b1c7-5dc293802878 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:22.229354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d67ec-8af3-471c-b1c7-5dc293802878 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:22.229649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d67ec-8af3-471c-b1c7-5dc293802878 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:22.230415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d67ec-8af3-471c-b1c7-5dc293802878 None admin] RBAC: Adding query filter params (interface=public, service_id=35d570ad6bcb45d6a2e71c12e7d2fe00, region_id=RegionOne) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:22.230723 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d67ec-8af3-471c-b1c7-5dc293802878 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:22.232983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d67ec-8af3-471c-b1c7-5dc293802878 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:22.244063 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 378/753] 10.222.0.36 () {56 vars in 1251 bytes} [Sun Oct 30 00:36:22 2022] GET /identity/v3/endpoints?service_id=35d570ad6bcb45d6a2e71c12e7d2fe00&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:22.245644 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722adefb-448a-4595-be4b-5e30c8c071eb None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:22.371893 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722adefb-448a-4595-be4b-5e30c8c071eb None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:22.374971 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9587e-3166-4c22-9aa5-0aa876fa37c8 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:22.375262 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9587e-3166-4c22-9aa5-0aa876fa37c8 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:22.375539 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9587e-3166-4c22-9aa5-0aa876fa37c8 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:22.377291 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 379/754] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:22.383201 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf8fd1-0f79-4c6e-b4f4-e4fe03c81fec None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:22.383512 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf8fd1-0f79-4c6e-b4f4-e4fe03c81fec None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:22.383755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf8fd1-0f79-4c6e-b4f4-e4fe03c81fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:22.405954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-b3cf8fd1-0f79-4c6e-b4f4-e4fe03c81fec None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:22.407182 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b3cf8fd1-0f79-4c6e-b4f4-e4fe03c81fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:22.408563 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:22.415319 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fe155-fd40-45af-8f59-40b93d58f9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:22.416007 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fe155-fd40-45af-8f59-40b93d58f9d6 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:22.416324 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fe155-fd40-45af-8f59-40b93d58f9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:22.416591 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fe155-fd40-45af-8f59-40b93d58f9d6 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:22.417249 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fe155-fd40-45af-8f59-40b93d58f9d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:22.420239 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fe155-fd40-45af-8f59-40b93d58f9d6 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:22.434130 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 376/755] 10.222.0.36 () {56 vars in 1087 bytes} [Sun Oct 30 00:36:22 2022] GET /identity/v3/services => generated 1957 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:36:22.440261 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:22.509791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 380/756] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:22 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:36:22.795302 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fe155-fd40-45af-8f59-40b93d58f9d6 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:22.924191 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fe155-fd40-45af-8f59-40b93d58f9d6 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:22.962151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f24c4a-c041-41e3-879e-105197221ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:22.962766 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f24c4a-c041-41e3-879e-105197221ce5 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:22.963039 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f24c4a-c041-41e3-879e-105197221ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:22.963306 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f24c4a-c041-41e3-879e-105197221ce5 None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:22.963961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f24c4a-c041-41e3-879e-105197221ce5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:22.965965 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f24c4a-c041-41e3-879e-105197221ce5 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:22.984146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.sql.core [None req-30f24c4a-c041-41e3-879e-105197221ce5 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 30 00:36:22.984146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 30 00:36:22.984146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [parameters: {'user_id': '8830f92ddb7540409ae9158e8684f6c2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 30 00:36:22.984146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88201) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 30 00:36:22.991613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-30f24c4a-c041-41e3-879e-105197221ce5 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 30 00:36:22.991613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:22.991613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:36:22.991613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:36:22.991613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:36:22.991613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:36:22.991613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:36:22.991613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:36:22.991613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:36:22.991613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:36:22.991613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:36:22.991613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:36:22.991613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:36:22.991613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application result.read() Oct 30 00:36:22.991613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:36:22.992666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:36:22.992666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:36:22.992666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:36:22.992666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:36:22.992666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:36:22.992666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:36:22.992666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:36:22.992666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 30 00:36:22.992666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:22.992666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 30 00:36:22.992666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:22.992666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:22.992666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 30 00:36:22.992666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 30 00:36:22.992666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 30 00:36:22.992666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 30 00:36:22.992666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:36:22.992666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:36:22.993763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 30 00:36:22.993763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application yield resource Oct 30 00:36:22.993763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:36:22.993763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:36:22.993763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 30 00:36:22.993763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.session.rollback() Oct 30 00:36:22.993763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 30 00:36:22.993763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.force_reraise() Oct 30 00:36:22.993763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 30 00:36:22.993763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise self.value Oct 30 00:36:22.993763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 30 00:36:22.993763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 30 00:36:22.993763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 30 00:36:22.993763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application session.commit() Oct 30 00:36:22.993763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 30 00:36:22.993763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 30 00:36:22.993763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 30 00:36:22.994765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._prepare_impl() Oct 30 00:36:22.994765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 30 00:36:22.994765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.session.flush() Oct 30 00:36:22.994765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 30 00:36:22.994765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._flush(objects) Oct 30 00:36:22.994765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 30 00:36:22.994765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 30 00:36:22.994765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 30 00:36:22.994765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application compat.raise_( Oct 30 00:36:22.994765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:36:22.994765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception Oct 30 00:36:22.994765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 30 00:36:22.994765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application flush_context.execute() Oct 30 00:36:22.994765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 30 00:36:22.994765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rec.execute(self) Oct 30 00:36:22.994765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 30 00:36:22.994765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 30 00:36:22.995761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 30 00:36:22.995761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 30 00:36:22.995761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 30 00:36:22.995761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 30 00:36:22.995761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 30 00:36:22.995761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 30 00:36:22.995761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 30 00:36:22.995761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 30 00:36:22.995761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 30 00:36:22.995761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 30 00:36:22.995761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 30 00:36:22.995761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 30 00:36:22.995761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 30 00:36:22.995761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 30 00:36:22.995761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:36:22.995761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception Oct 30 00:36:22.997381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:36:22.997381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:36:22.997381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:36:22.997381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:36:22.997381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:36:22.997381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:36:22.997381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:36:22.997381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:36:22.997381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:36:22.997381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:36:22.997381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:36:22.997381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application result.read() Oct 30 00:36:22.997381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:36:22.997381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:36:22.997381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:36:22.997381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:36:22.998379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:36:22.998379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:36:22.998379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:36:22.998379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:36:22.998379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 30 00:36:22.998379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 30 00:36:22.998379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [parameters: {'user_id': '8830f92ddb7540409ae9158e8684f6c2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 30 00:36:22.998379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 30 00:36:22.998379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:22.998379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:22.998379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:22.998379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:22.998379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:22.998379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:22.998379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:22.999272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:22.999272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:22.999272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:22.999272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:22.999272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:22.999272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:22.999272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:22.999272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 30 00:36:22.999272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 30 00:36:22.999272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:22.999272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:22.999272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:36:22.999272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:36:22.999272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:36:22.999272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:36:22.999272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 30 00:36:22.999272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 30 00:36:23.002012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 30 00:36:23.002012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 30 00:36:23.002012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 30 00:36:23.002012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 30 00:36:23.002012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 30 00:36:23.002012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:23.002012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 377/757] 10.222.0.36 () {60 vars in 1131 bytes} [Sun Oct 30 00:36:22 2022] POST /identity/v3/users => generated 160 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:23.002913 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf8fd1-0f79-4c6e-b4f4-e4fe03c81fec None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:23.125913 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf8fd1-0f79-4c6e-b4f4-e4fe03c81fec None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:23.164924 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537827ba-7f35-4d3d-ae18-b4a88b151c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:23.165569 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537827ba-7f35-4d3d-ae18-b4a88b151c10 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:23.165849 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537827ba-7f35-4d3d-ae18-b4a88b151c10 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:23.166121 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537827ba-7f35-4d3d-ae18-b4a88b151c10 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:23.175013 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537827ba-7f35-4d3d-ae18-b4a88b151c10 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:23.176572 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537827ba-7f35-4d3d-ae18-b4a88b151c10 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:23.189649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-537827ba-7f35-4d3d-ae18-b4a88b151c10 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 30 00:36:23.189649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:23.189649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:23.189649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:23.189649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:23.189649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:23.189649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:23.189649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:23.189649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:23.189649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:23.189649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:23.189649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:23.189649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:36:23.189649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:36:23.189649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:36:23.189649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:36:23.190694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:23.190694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:23.190694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:36:23.190694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:36:23.190694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:36:23.190694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:36:23.190694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:23.190694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:23.190694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:23.190694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:23.190694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:23.190694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:23.190694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:23.190694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:23.190694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:23.190694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:23.190694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:23.190694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:23.191740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:23.191740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:23.191740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:36:23.191740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:36:23.191740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:36:23.191740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:36:23.191740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:36:23.191740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:36:23.191740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 30 00:36:23.191740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:23.191740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 381/758] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:36:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:23.197666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f24c4a-c041-41e3-879e-105197221ce5 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:23.272514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:23.320871 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819389d-4781-4e2a-beb8-3f542d3954bb None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:23.321194 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819389d-4781-4e2a-beb8-3f542d3954bb None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:23.321400 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819389d-4781-4e2a-beb8-3f542d3954bb None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:23.323248 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 382/759] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:23.328707 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe206eb0-3237-4a50-81f4-f8ffbba3b07b None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:23.329041 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe206eb0-3237-4a50-81f4-f8ffbba3b07b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:23.329263 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe206eb0-3237-4a50-81f4-f8ffbba3b07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:23.334879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f24c4a-c041-41e3-879e-105197221ce5 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:23.348965 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-fe206eb0-3237-4a50-81f4-f8ffbba3b07b None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:23.350456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-fe206eb0-3237-4a50-81f4-f8ffbba3b07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:23.353629 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:23.379561 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0b986-3589-4b95-b8fa-bcee2c4157d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:23.380983 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0b986-3589-4b95-b8fa-bcee2c4157d9 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:23.381704 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0b986-3589-4b95-b8fa-bcee2c4157d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:23.382339 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0b986-3589-4b95-b8fa-bcee2c4157d9 None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:23.384151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0b986-3589-4b95-b8fa-bcee2c4157d9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:23.384830 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0b986-3589-4b95-b8fa-bcee2c4157d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:23.387896 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0b986-3589-4b95-b8fa-bcee2c4157d9 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:23.445859 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 378/760] 10.222.0.36 () {56 vars in 1101 bytes} [Sun Oct 30 00:36:23 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:23.446511 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 383/761] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:23 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:36:23.717506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0b986-3589-4b95-b8fa-bcee2c4157d9 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:23.853894 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0b986-3589-4b95-b8fa-bcee2c4157d9 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:23.874991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5babb3e0-5ef5-4e65-a65a-5667997c12df None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:23.874991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5babb3e0-5ef5-4e65-a65a-5667997c12df None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:23.875171 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5babb3e0-5ef5-4e65-a65a-5667997c12df None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:23.878125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:23.881358 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 384/762] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:23.882741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaada9c-ff2c-4310-b740-5dec56526b5e None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:23.883023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaada9c-ff2c-4310-b740-5dec56526b5e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:23.883294 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaada9c-ff2c-4310-b740-5dec56526b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:23.909519 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-cfaada9c-ff2c-4310-b740-5dec56526b5e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:23.910733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-cfaada9c-ff2c-4310-b740-5dec56526b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:23.910991 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce0108-8208-4464-94c5-9f7f4d8cf367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:23.911637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce0108-8208-4464-94c5-9f7f4d8cf367 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:23.911915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce0108-8208-4464-94c5-9f7f4d8cf367 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:23.912228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce0108-8208-4464-94c5-9f7f4d8cf367 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:23.912818 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce0108-8208-4464-94c5-9f7f4d8cf367 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:23.914273 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce0108-8208-4464-94c5-9f7f4d8cf367 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:23.924662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-94ce0108-8208-4464-94c5-9f7f4d8cf367 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 30 00:36:23.924662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:23.924662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:23.924662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:23.924662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:23.924662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:23.924662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:23.924662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:23.924662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 30 00:36:23.924662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:23.924662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:23.924662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:23.924662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:23.924662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:23.924662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:23.924662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:23.924662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:23.925719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:23.925719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:23.925719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:23.925719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:23.925719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:23.925719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:23.925719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:23.925719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:23.925719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:23.925719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:23.925719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:23.925719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:23.925719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:23.925719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:23.925719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:23.925719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:23.926545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:23.926545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:23.926545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:23.926545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:23.926545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:23.926545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:23.926545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:23.926545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:23.926545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:23.926545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:23.926545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:23.926545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:23.926545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 30 00:36:23.926545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:23.930923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 379/763] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:36:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:23.931832 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce0108-8208-4464-94c5-9f7f4d8cf367 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:23.988851 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 385/764] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:23 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:24.052599 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce0108-8208-4464-94c5-9f7f4d8cf367 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:24.088610 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe97f9b-3859-4a76-b8d9-942d7f372540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:24.089220 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe97f9b-3859-4a76-b8d9-942d7f372540 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:24.089496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe97f9b-3859-4a76-b8d9-942d7f372540 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:24.089766 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe97f9b-3859-4a76-b8d9-942d7f372540 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:24.090401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe97f9b-3859-4a76-b8d9-942d7f372540 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:24.090736 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe97f9b-3859-4a76-b8d9-942d7f372540 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:24.092239 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe97f9b-3859-4a76-b8d9-942d7f372540 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:24.097716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 380/765] 10.222.0.36 () {56 vars in 1112 bytes} [Sun Oct 30 00:36:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:24.109236 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaada9c-ff2c-4310-b740-5dec56526b5e None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:24.187549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:24.228495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaada9c-ff2c-4310-b740-5dec56526b5e None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:24.260167 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe97f9b-3859-4a76-b8d9-942d7f372540 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:24.273675 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce387a04-2157-4942-81f1-f610fe2e6565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:24.274292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce387a04-2157-4942-81f1-f610fe2e6565 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:24.274569 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce387a04-2157-4942-81f1-f610fe2e6565 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:24.274836 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce387a04-2157-4942-81f1-f610fe2e6565 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:24.275470 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce387a04-2157-4942-81f1-f610fe2e6565 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:24.275769 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce387a04-2157-4942-81f1-f610fe2e6565 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:24.277208 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce387a04-2157-4942-81f1-f610fe2e6565 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:24.284012 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 386/766] 10.222.0.36 () {56 vars in 1112 bytes} [Sun Oct 30 00:36:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:24.289818 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce387a04-2157-4942-81f1-f610fe2e6565 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:24.319369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:24.388464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe97f9b-3859-4a76-b8d9-942d7f372540 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:24.411264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce387a04-2157-4942-81f1-f610fe2e6565 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:24.435590 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a4012-d3f4-4b08-b061-588a00ea391f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:24.436902 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a4012-d3f4-4b08-b061-588a00ea391f None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:24.437393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a4012-d3f4-4b08-b061-588a00ea391f None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:24.437814 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a4012-d3f4-4b08-b061-588a00ea391f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:24.438415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a4012-d3f4-4b08-b061-588a00ea391f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:24.439804 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a4012-d3f4-4b08-b061-588a00ea391f None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:24.449106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-b00a4012-d3f4-4b08-b061-588a00ea391f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:36:24.449106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:24.449106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:24.449106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:24.449106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:24.449106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:24.449106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:24.449106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:24.449106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:36:24.449106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:24.449106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:24.449106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:24.449106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:24.449106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:24.449106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:24.449106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:24.449106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:24.450715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:24.450715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:24.450715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:24.450715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:24.450715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:24.450715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:24.450715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:24.450715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:24.450715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:24.450715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:24.450715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:24.450715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:24.450715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:24.450715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:24.450715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:24.450715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:24.451614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:24.451614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:24.451614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:24.451614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:24.451614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:24.451614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:24.451614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:24.451614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:24.451614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:24.451614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:24.451614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:24.451614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:24.451614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 30 00:36:24.451614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:24.456575 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 381/767] 10.222.0.36 () {56 vars in 1126 bytes} [Sun Oct 30 00:36:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:36:24.460479 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a4012-d3f4-4b08-b061-588a00ea391f None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:24.461984 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfe173-b2cb-4027-ae25-9067c95655b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:24.462604 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfe173-b2cb-4027-ae25-9067c95655b5 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:24.462879 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfe173-b2cb-4027-ae25-9067c95655b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:24.463148 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfe173-b2cb-4027-ae25-9067c95655b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:24.463848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfe173-b2cb-4027-ae25-9067c95655b5 None admin] RBAC: Adding query filter params (interface=public, service_id=44aab7a1ad7541f59ed6e9ddf3dc07b9, region_id=RegionOne) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:24.468347 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfe173-b2cb-4027-ae25-9067c95655b5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:24.470235 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfe173-b2cb-4027-ae25-9067c95655b5 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:24.472009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:24.481088 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 387/768] 10.222.0.36 () {56 vars in 1251 bytes} [Sun Oct 30 00:36:24 2022] GET /identity/v3/endpoints?service_id=44aab7a1ad7541f59ed6e9ddf3dc07b9&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:24.482136 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfe173-b2cb-4027-ae25-9067c95655b5 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:24.590927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a4012-d3f4-4b08-b061-588a00ea391f None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:24.599853 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfe173-b2cb-4027-ae25-9067c95655b5 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:24.603821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:24.632637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553393b-6354-4b45-9b36-d091df8e7c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:24.633633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553393b-6354-4b45-9b36-d091df8e7c41 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:24.634353 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553393b-6354-4b45-9b36-d091df8e7c41 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:24.635253 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553393b-6354-4b45-9b36-d091df8e7c41 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:24.636394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553393b-6354-4b45-9b36-d091df8e7c41 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:24.636924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553393b-6354-4b45-9b36-d091df8e7c41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:24.639116 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553393b-6354-4b45-9b36-d091df8e7c41 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:24.645609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 382/769] 10.222.0.36 () {56 vars in 1122 bytes} [Sun Oct 30 00:36:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:24.645860 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cc2b7-aaa3-4d0a-b06f-abbe73f72d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:24.646577 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cc2b7-aaa3-4d0a-b06f-abbe73f72d19 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:24.646965 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cc2b7-aaa3-4d0a-b06f-abbe73f72d19 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:24.647249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cc2b7-aaa3-4d0a-b06f-abbe73f72d19 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:24.647867 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605cc2b7-aaa3-4d0a-b06f-abbe73f72d19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:24.649327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605cc2b7-aaa3-4d0a-b06f-abbe73f72d19 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:24.654035 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553393b-6354-4b45-9b36-d091df8e7c41 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:24.665699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 388/770] 10.222.0.36 () {56 vars in 1087 bytes} [Sun Oct 30 00:36:24 2022] GET /identity/v3/services => generated 1957 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:36:24.786958 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553393b-6354-4b45-9b36-d091df8e7c41 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:24.834843 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c9f19-5c50-45ca-8e1b-c2daebe7cbcf None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:24.835132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c9f19-5c50-45ca-8e1b-c2daebe7cbcf None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:24.835408 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c9f19-5c50-45ca-8e1b-c2daebe7cbcf None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:24.836930 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 389/771] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:36:24.838163 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1f93e-b1dc-4d02-8e35-7609159277f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:24.838800 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1f93e-b1dc-4d02-8e35-7609159277f9 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:24.839083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1f93e-b1dc-4d02-8e35-7609159277f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:24.840371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1f93e-b1dc-4d02-8e35-7609159277f9 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:24.840371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a1f93e-b1dc-4d02-8e35-7609159277f9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:24.840371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a1f93e-b1dc-4d02-8e35-7609159277f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:24.842171 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a1f93e-b1dc-4d02-8e35-7609159277f9 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:24.843168 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7464b41-68a2-4081-a7bf-c24cd54581c7 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:24.843441 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7464b41-68a2-4081-a7bf-c24cd54581c7 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:24.843692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7464b41-68a2-4081-a7bf-c24cd54581c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:24.854926 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 383/772] 10.222.0.36 () {56 vars in 1122 bytes} [Sun Oct 30 00:36:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:24.861506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1f93e-b1dc-4d02-8e35-7609159277f9 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:24.871465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-a7464b41-68a2-4081-a7bf-c24cd54581c7 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:24.873507 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a7464b41-68a2-4081-a7bf-c24cd54581c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:24.915863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:24.977008 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:24.992944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 390/773] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:24 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:24.994705 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1f93e-b1dc-4d02-8e35-7609159277f9 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:25.034180 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d903215-4dd9-480e-8e82-b158385ee1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:25.034800 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d903215-4dd9-480e-8e82-b158385ee1b6 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:25.035080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d903215-4dd9-480e-8e82-b158385ee1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:25.035354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d903215-4dd9-480e-8e82-b158385ee1b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:25.036033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d903215-4dd9-480e-8e82-b158385ee1b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:25.037493 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d903215-4dd9-480e-8e82-b158385ee1b6 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:25.061208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 384/774] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:36:24 2022] POST /identity/v3/endpoints => generated 325 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:36:25.272038 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7464b41-68a2-4081-a7bf-c24cd54581c7 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:25.396646 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7464b41-68a2-4081-a7bf-c24cd54581c7 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:25.445317 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d812626-93c2-4dec-ab49-fc204ca9b5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:25.445960 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d812626-93c2-4dec-ab49-fc204ca9b5f7 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:25.446242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d812626-93c2-4dec-ab49-fc204ca9b5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:25.446517 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d812626-93c2-4dec-ab49-fc204ca9b5f7 None admin] PATH_INFO: `/v3/services/image` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:25.447097 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d812626-93c2-4dec-ab49-fc204ca9b5f7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:25.448524 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d812626-93c2-4dec-ab49-fc204ca9b5f7 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:25.458331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-8d812626-93c2-4dec-ab49-fc204ca9b5f7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 30 00:36:25.458331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:25.458331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:25.458331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:25.458331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:25.458331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:25.458331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:25.458331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:25.458331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 30 00:36:25.458331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:25.458331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:25.458331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:25.458331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:25.458331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:25.458331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:25.458331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:25.458331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:25.459437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:25.459437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:25.459437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:25.459437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:25.459437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:25.459437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:25.459437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:25.459437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:25.459437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:25.459437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:25.459437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:25.459437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:25.459437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:25.459437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:25.459437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:25.459437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:25.460369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:25.460369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:25.460369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:25.460369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:25.460369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:25.460369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:25.460369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:25.460369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:25.460369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:25.460369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:25.460369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:25.460369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:25.460369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 30 00:36:25.460369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:25.460369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 391/775] 10.222.0.36 () {56 vars in 1105 bytes} [Sun Oct 30 00:36:25 2022] GET /identity/v3/services/image => generated 86 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:25.465618 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d903215-4dd9-480e-8e82-b158385ee1b6 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:25.585336 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:25.596159 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d903215-4dd9-480e-8e82-b158385ee1b6 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:25.637446 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9a23f-a59c-4d9d-bf90-aeb94bb1cc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:25.639041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9a23f-a59c-4d9d-bf90-aeb94bb1cc42 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:25.639041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9a23f-a59c-4d9d-bf90-aeb94bb1cc42 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:25.639326 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9a23f-a59c-4d9d-bf90-aeb94bb1cc42 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:25.639971 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9a23f-a59c-4d9d-bf90-aeb94bb1cc42 None admin] RBAC: Adding query filter params (name=image) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:25.640301 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9a23f-a59c-4d9d-bf90-aeb94bb1cc42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:25.641718 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9a23f-a59c-4d9d-bf90-aeb94bb1cc42 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:25.651779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 385/776] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:25.654024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d812626-93c2-4dec-ab49-fc204ca9b5f7 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:25.720252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:25.780458 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d812626-93c2-4dec-ab49-fc204ca9b5f7 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:25.831779 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6be895-988b-4a05-991c-9df051a34c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:25.832602 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6be895-988b-4a05-991c-9df051a34c13 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:25.832928 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6be895-988b-4a05-991c-9df051a34c13 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:25.833215 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6be895-988b-4a05-991c-9df051a34c13 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:25.833844 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6be895-988b-4a05-991c-9df051a34c13 None admin] RBAC: Adding query filter params (type=image) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:25.834142 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6be895-988b-4a05-991c-9df051a34c13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:25.835579 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6be895-988b-4a05-991c-9df051a34c13 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:25.846467 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 392/777] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:25.847589 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9a23f-a59c-4d9d-bf90-aeb94bb1cc42 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:25.976564 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9a23f-a59c-4d9d-bf90-aeb94bb1cc42 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:26.017780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9cdff-a06e-410d-a393-0c9c067b1f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:26.018407 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9cdff-a06e-410d-a393-0c9c067b1f04 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:26.018685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9cdff-a06e-410d-a393-0c9c067b1f04 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:26.018957 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9cdff-a06e-410d-a393-0c9c067b1f04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:26.019650 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9cdff-a06e-410d-a393-0c9c067b1f04 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:26.021059 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9cdff-a06e-410d-a393-0c9c067b1f04 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:26.048387 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 386/778] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:36:25 2022] POST /identity/v3/endpoints => generated 327 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:36:26.920075 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c15582-7770-42d1-bc51-99fca98d4796 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:26.920638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c15582-7770-42d1-bc51-99fca98d4796 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:26.920638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c15582-7770-42d1-bc51-99fca98d4796 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:26.922216 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 393/779] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:26.927665 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97022e13-e1ec-4c20-80a7-19fe5212b35e None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:26.928179 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97022e13-e1ec-4c20-80a7-19fe5212b35e None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:26.928281 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97022e13-e1ec-4c20-80a7-19fe5212b35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:26.947956 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-97022e13-e1ec-4c20-80a7-19fe5212b35e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:26.949550 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-97022e13-e1ec-4c20-80a7-19fe5212b35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:26.962737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:27.045421 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 387/780] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:26 2022] POST /identity/v3/auth/tokens => generated 1962 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:27.323571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c15582-7770-42d1-bc51-99fca98d4796 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:27.349534 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfb375-e241-4fc7-8aed-d3d6eac9fff9 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:27.349823 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfb375-e241-4fc7-8aed-d3d6eac9fff9 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:27.350112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfb375-e241-4fc7-8aed-d3d6eac9fff9 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:27.351724 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 388/781] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:36:27.357738 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af45b0-f5d1-4d56-806c-877d3d5d08cc None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:27.358166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af45b0-f5d1-4d56-806c-877d3d5d08cc None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:27.358284 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af45b0-f5d1-4d56-806c-877d3d5d08cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:27.380436 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-b0af45b0-f5d1-4d56-806c-877d3d5d08cc None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:27.382109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-b0af45b0-f5d1-4d56-806c-877d3d5d08cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:27.449553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c15582-7770-42d1-bc51-99fca98d4796 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:27.469139 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:27.476131 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 389/782] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:27 2022] POST /identity/v3/auth/tokens => generated 1962 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:36:27.493178 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4c24a-b3a5-496a-af26-9f98be206e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:27.493926 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4c24a-b3a5-496a-af26-9f98be206e1c None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:27.494253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4c24a-b3a5-496a-af26-9f98be206e1c None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:27.494588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4c24a-b3a5-496a-af26-9f98be206e1c None admin] PATH_INFO: `/v3/services/compute` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:27.495236 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4c24a-b3a5-496a-af26-9f98be206e1c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:27.496717 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4c24a-b3a5-496a-af26-9f98be206e1c None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:27.506742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-dae4c24a-b3a5-496a-af26-9f98be206e1c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 30 00:36:27.506742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:27.506742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:36:27.506742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:36:27.506742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:36:27.506742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:36:27.506742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:36:27.506742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:27.506742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 30 00:36:27.506742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:27.506742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:27.506742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:27.506742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:27.506742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:27.506742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:27.506742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:27.506742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:27.507784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:27.507784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:27.507784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:27.507784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:27.507784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:27.507784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:27.507784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:36:27.507784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:36:27.507784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:36:27.507784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:36:27.507784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:27.507784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:27.507784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:27.507784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:27.507784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:27.507784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:27.509009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:27.509009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:27.509009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:27.509009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:27.509009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:27.509009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:27.509009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:27.509009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:27.509009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:27.509009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:27.509009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:36:27.509009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:36:27.509009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 30 00:36:27.509009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:27.509009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 394/783] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:36:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:27.513936 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af45b0-f5d1-4d56-806c-877d3d5d08cc None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:27.594041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:27.629258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af45b0-f5d1-4d56-806c-877d3d5d08cc None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:27.668271 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262bb2a-fe08-4994-8e9c-a0f825dd9649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:27.669122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262bb2a-fe08-4994-8e9c-a0f825dd9649 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:27.670055 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262bb2a-fe08-4994-8e9c-a0f825dd9649 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:27.670343 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262bb2a-fe08-4994-8e9c-a0f825dd9649 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:27.670959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262bb2a-fe08-4994-8e9c-a0f825dd9649 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:27.671257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262bb2a-fe08-4994-8e9c-a0f825dd9649 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:27.672659 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262bb2a-fe08-4994-8e9c-a0f825dd9649 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:27.682545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 390/784] 10.222.0.36 () {56 vars in 1112 bytes} [Sun Oct 30 00:36:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:27.683499 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4c24a-b3a5-496a-af26-9f98be206e1c None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:27.744331 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262bb2a-fe08-4994-8e9c-a0f825dd9649 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:27.798260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4c24a-b3a5-496a-af26-9f98be206e1c None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:27.840520 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc893d8-a7b1-446a-b206-e01bf9b229fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:27.841903 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc893d8-a7b1-446a-b206-e01bf9b229fa None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:27.842612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc893d8-a7b1-446a-b206-e01bf9b229fa None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:27.842884 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc893d8-a7b1-446a-b206-e01bf9b229fa None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:27.843529 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc893d8-a7b1-446a-b206-e01bf9b229fa None admin] RBAC: Adding query filter params (type=compute) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:27.843819 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc893d8-a7b1-446a-b206-e01bf9b229fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:27.845672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc893d8-a7b1-446a-b206-e01bf9b229fa None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:27.856144 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 395/785] 10.222.0.36 () {56 vars in 1112 bytes} [Sun Oct 30 00:36:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:27.861555 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc893d8-a7b1-446a-b206-e01bf9b229fa None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:27.875342 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262bb2a-fe08-4994-8e9c-a0f825dd9649 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:27.914149 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc17e3e-48a3-47fe-9406-3ab52075f9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:27.914807 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc17e3e-48a3-47fe-9406-3ab52075f9d6 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:27.915079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc17e3e-48a3-47fe-9406-3ab52075f9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:27.915352 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc17e3e-48a3-47fe-9406-3ab52075f9d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:27.916506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc17e3e-48a3-47fe-9406-3ab52075f9d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:27.917947 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc17e3e-48a3-47fe-9406-3ab52075f9d6 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:27.924585 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 391/786] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:27.927322 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc17e3e-48a3-47fe-9406-3ab52075f9d6 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:27.928796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:27.932166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:27.999879 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc893d8-a7b1-446a-b206-e01bf9b229fa None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:28.039563 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf852a0-a7bf-437e-9a09-27baa421ec98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:28.040319 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf852a0-a7bf-437e-9a09-27baa421ec98 None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:28.040604 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf852a0-a7bf-437e-9a09-27baa421ec98 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:28.040877 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf852a0-a7bf-437e-9a09-27baa421ec98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:28.041552 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf852a0-a7bf-437e-9a09-27baa421ec98 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:28.042969 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf852a0-a7bf-437e-9a09-27baa421ec98 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:28.048005 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc17e3e-48a3-47fe-9406-3ab52075f9d6 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:28.067269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 396/787] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:36:27 2022] POST /identity/v3/endpoints => generated 334 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:36:28.109893 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc46346-c834-4d8c-a266-5645ef9b926a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:28.111004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc46346-c834-4d8c-a266-5645ef9b926a None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:28.111540 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc46346-c834-4d8c-a266-5645ef9b926a None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:28.112001 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc46346-c834-4d8c-a266-5645ef9b926a None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:28.112929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc46346-c834-4d8c-a266-5645ef9b926a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:28.114653 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc46346-c834-4d8c-a266-5645ef9b926a None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:28.148889 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 392/788] 10.222.0.36 () {60 vars in 1141 bytes} [Sun Oct 30 00:36:27 2022] POST /identity/v3/projects => generated 311 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:36:28.376754 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c7966-2eb0-47dd-a494-a71df2e4a2ca None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:28.376963 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c7966-2eb0-47dd-a494-a71df2e4a2ca None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:28.377237 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c7966-2eb0-47dd-a494-a71df2e4a2ca None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:28.378858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 397/789] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:28.389447 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd7918-feeb-44e4-8534-ad0cb1251fd2 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:28.389447 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd7918-feeb-44e4-8534-ad0cb1251fd2 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:28.389447 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd7918-feeb-44e4-8534-ad0cb1251fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:28.404716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-a6bd7918-feeb-44e4-8534-ad0cb1251fd2 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:28.406447 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a6bd7918-feeb-44e4-8534-ad0cb1251fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:28.460773 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:28.484923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 393/790] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:28.760990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c7966-2eb0-47dd-a494-a71df2e4a2ca None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:28.871139 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:28.876921 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c7966-2eb0-47dd-a494-a71df2e4a2ca None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:28.926467 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb053993-8778-4f36-ab72-46040d8767f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:28.927093 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb053993-8778-4f36-ab72-46040d8767f6 None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:28.927374 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb053993-8778-4f36-ab72-46040d8767f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:28.927645 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb053993-8778-4f36-ab72-46040d8767f6 None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:28.928357 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb053993-8778-4f36-ab72-46040d8767f6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:28.929741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb053993-8778-4f36-ab72-46040d8767f6 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:28.939149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.sql.core [None req-eb053993-8778-4f36-ab72-46040d8767f6 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 30 00:36:28.939149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 30 00:36:28.939149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [parameters: {'id': 'e1fa3593d919453688c4de5647bde377', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 30 00:36:28.939149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88200) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 30 00:36:28.956699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-eb053993-8778-4f36-ab72-46040d8767f6 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 30 00:36:28.956699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:28.956699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:36:28.956699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:36:28.956699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:36:28.956699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:36:28.956699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:36:28.956699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:36:28.956699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:36:28.956699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:36:28.956699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:36:28.956699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:36:28.956699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:36:28.956699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application result.read() Oct 30 00:36:28.957669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:36:28.957669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:36:28.957669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:36:28.957669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:36:28.957669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:36:28.957669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:36:28.957669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:36:28.957669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:36:28.957669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 30 00:36:28.957669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:28.957669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 30 00:36:28.957669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:28.957669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:28.957669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 30 00:36:28.957669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 30 00:36:28.957669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 30 00:36:28.957669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 30 00:36:28.957669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:36:28.958651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:36:28.958651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 30 00:36:28.958651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application yield resource Oct 30 00:36:28.958651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 30 00:36:28.958651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application next(self.gen) Oct 30 00:36:28.958651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 30 00:36:28.958651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self.session.rollback() Oct 30 00:36:28.958651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 30 00:36:28.958651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self.force_reraise() Oct 30 00:36:28.958651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 30 00:36:28.958651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise self.value Oct 30 00:36:28.958651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 30 00:36:28.958651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 30 00:36:28.958651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 30 00:36:28.958651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application session.commit() Oct 30 00:36:28.958651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 30 00:36:28.958651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 30 00:36:28.961698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 30 00:36:28.961698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._prepare_impl() Oct 30 00:36:28.961698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 30 00:36:28.961698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self.session.flush() Oct 30 00:36:28.961698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 30 00:36:28.961698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._flush(objects) Oct 30 00:36:28.961698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 30 00:36:28.961698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 30 00:36:28.961698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 30 00:36:28.961698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application compat.raise_( Oct 30 00:36:28.961698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:36:28.961698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception Oct 30 00:36:28.961698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 30 00:36:28.961698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application flush_context.execute() Oct 30 00:36:28.961698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 30 00:36:28.961698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rec.execute(self) Oct 30 00:36:28.961698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 30 00:36:28.964350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 30 00:36:28.964350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 30 00:36:28.964350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 30 00:36:28.964350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 30 00:36:28.964350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 30 00:36:28.964350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 30 00:36:28.964350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 30 00:36:28.964350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 30 00:36:28.964350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 30 00:36:28.964350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 30 00:36:28.964350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 30 00:36:28.964350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 30 00:36:28.964350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 30 00:36:28.964350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 30 00:36:28.964350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 30 00:36:28.966210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 30 00:36:28.966210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception Oct 30 00:36:28.966210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 30 00:36:28.966210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 30 00:36:28.966210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 30 00:36:28.966210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 30 00:36:28.966210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 30 00:36:28.966210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application result = self._query(query) Oct 30 00:36:28.966210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 30 00:36:28.966210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application conn.query(q) Oct 30 00:36:28.966210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 30 00:36:28.966210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 30 00:36:28.966210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 30 00:36:28.966210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application result.read() Oct 30 00:36:28.966210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 30 00:36:28.966210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 30 00:36:28.966210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 30 00:36:28.967905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 30 00:36:28.967905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 30 00:36:28.967905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 30 00:36:28.967905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 30 00:36:28.967905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 30 00:36:28.967905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 30 00:36:28.967905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 30 00:36:28.967905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [parameters: {'id': 'e1fa3593d919453688c4de5647bde377', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 30 00:36:28.967905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 30 00:36:28.967905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:28.967905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:28.967905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:28.967905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:28.967905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 30 00:36:28.967905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 30 00:36:28.973521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 30 00:36:28.973521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 30 00:36:28.973521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 30 00:36:28.973521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:28.973521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:36:28.973521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:28.973521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:28.973521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:28.973521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:28.973521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:28.973521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:28.973521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:28.973521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:28.973521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:28.973521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:28.973521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:28.973521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:28.975536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 30 00:36:28.975536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 30 00:36:28.975536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:28.975536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:28.975536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 30 00:36:28.975536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 30 00:36:28.975536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:28.975536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:28.975536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 30 00:36:28.975536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 30 00:36:28.975536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 30 00:36:28.975536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:28.975536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 398/791] 10.222.0.36 () {60 vars in 1137 bytes} [Sun Oct 30 00:36:28 2022] POST /identity/v3/domains => generated 190 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:28.976967 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd7918-feeb-44e4-8534-ad0cb1251fd2 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:29.093364 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd7918-feeb-44e4-8534-ad0cb1251fd2 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:29.139376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734fb1c-11db-4edf-a8ac-277e99d6581e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:29.140373 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734fb1c-11db-4edf-a8ac-277e99d6581e None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:29.140802 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734fb1c-11db-4edf-a8ac-277e99d6581e None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:29.141243 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734fb1c-11db-4edf-a8ac-277e99d6581e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:29.155127 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734fb1c-11db-4edf-a8ac-277e99d6581e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:29.156697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734fb1c-11db-4edf-a8ac-277e99d6581e None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:29.174757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-2734fb1c-11db-4edf-a8ac-277e99d6581e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:29.174757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:29.174757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:29.174757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:29.174757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:29.174757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:29.174757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:29.174757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:29.174757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:29.174757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:29.174757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:29.174757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:29.174757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:36:29.174757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:36:29.174757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:36:29.174757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:36:29.175815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:29.175815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:29.175815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:29.175815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:29.175815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:29.175815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:29.175815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:29.175815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:29.175815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:29.175815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:29.175815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:29.175815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:29.175815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:29.175815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:29.175815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:29.175815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:29.175815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:36:29.181105 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:36:29.181105 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:29.181105 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:29.181105 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 394/792] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:29.183869 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb053993-8778-4f36-ab72-46040d8767f6 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:29.297764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb053993-8778-4f36-ab72-46040d8767f6 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:29.336530 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41bc9fb-3f74-4cb4-baae-b7b96f7a9f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:29.337142 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41bc9fb-3f74-4cb4-baae-b7b96f7a9f10 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:29.337415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41bc9fb-3f74-4cb4-baae-b7b96f7a9f10 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:29.337746 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41bc9fb-3f74-4cb4-baae-b7b96f7a9f10 None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:29.338395 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41bc9fb-3f74-4cb4-baae-b7b96f7a9f10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:29.338757 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41bc9fb-3f74-4cb4-baae-b7b96f7a9f10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:29.340253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41bc9fb-3f74-4cb4-baae-b7b96f7a9f10 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:29.360206 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 399/793] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:30.454118 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70adb9b5-1124-4e6b-9db5-c0894b0fb798 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:30.454683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70adb9b5-1124-4e6b-9db5-c0894b0fb798 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:30.454683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70adb9b5-1124-4e6b-9db5-c0894b0fb798 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:30.456095 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 395/794] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:30.461647 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694c60b-15fd-46b6-a2e8-26bc5eea562f None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:30.462124 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694c60b-15fd-46b6-a2e8-26bc5eea562f None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:30.462215 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694c60b-15fd-46b6-a2e8-26bc5eea562f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:30.463941 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5abef4-3503-471c-a180-d5828da06b8f None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:30.464257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5abef4-3503-471c-a180-d5828da06b8f None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:30.464532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5abef4-3503-471c-a180-d5828da06b8f None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:30.465973 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 396/795] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:36:30.472410 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf8e81-191c-4672-9c20-f51830655cc7 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:30.472857 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf8e81-191c-4672-9c20-f51830655cc7 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:30.473271 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf8e81-191c-4672-9c20-f51830655cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:30.480547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-e694c60b-15fd-46b6-a2e8-26bc5eea562f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:30.482260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e694c60b-15fd-46b6-a2e8-26bc5eea562f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:30.495833 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-e6bf8e81-191c-4672-9c20-f51830655cc7 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:30.497046 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e6bf8e81-191c-4672-9c20-f51830655cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:30.502530 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:30.526610 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:30.559766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 400/796] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:30.573513 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 397/797] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:30.834547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694c60b-15fd-46b6-a2e8-26bc5eea562f None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:30.847727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf8e81-191c-4672-9c20-f51830655cc7 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:30.957909 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694c60b-15fd-46b6-a2e8-26bc5eea562f None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:30.960568 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf8e81-191c-4672-9c20-f51830655cc7 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:30.996102 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9de55-7bc2-452b-81a0-38593d2630ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:30.996773 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9de55-7bc2-452b-81a0-38593d2630ca None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:30.997052 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9de55-7bc2-452b-81a0-38593d2630ca None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:30.997322 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9de55-7bc2-452b-81a0-38593d2630ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:30.998260 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afd740-b1b1-4437-8401-bf6948361783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:30.998902 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afd740-b1b1-4437-8401-bf6948361783 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:30.999081 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb9de55-7bc2-452b-81a0-38593d2630ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:30.999199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afd740-b1b1-4437-8401-bf6948361783 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:30.999468 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afd740-b1b1-4437-8401-bf6948361783 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:31.000531 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb9de55-7bc2-452b-81a0-38593d2630ca None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:31.002118 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 401/798] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:31.007998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9de55-7bc2-452b-81a0-38593d2630ca None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:31.012675 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75afd740-b1b1-4437-8401-bf6948361783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:36:31.017516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-75afd740-b1b1-4437-8401-bf6948361783 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 30 00:36:31.017516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:31.017516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:31.017516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:31.017516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:31.017516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:31.017516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:31.017516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:31.017516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:31.017516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:31.017516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:31.017516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:31.017516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:36:31.017516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:36:31.017516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:36:31.017516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:36:31.018551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:36:31.018551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:36:31.018551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:36:31.018551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:36:31.018551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:31.018551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:31.018551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:31.018551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:31.018551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:31.018551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:31.018551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:31.018551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:31.018551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:31.018551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:31.018551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:31.018551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:31.019402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:31.019402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:31.019402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:31.019402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:31.019402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:36:31.019402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:36:31.019402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:36:31.019402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:36:31.019402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:36:31.019402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:36:31.019402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 30 00:36:31.019402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:31.019402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 398/799] 10.222.0.36 () {56 vars in 1120 bytes} [Sun Oct 30 00:36:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:36:31.024576 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afd740-b1b1-4437-8401-bf6948361783 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:31.036412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:31.051978 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:31.119518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9de55-7bc2-452b-81a0-38593d2630ca None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:31.134926 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afd740-b1b1-4437-8401-bf6948361783 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:31.155716 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279b693-2f25-4c9e-8ad4-1825595f2b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:31.156364 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279b693-2f25-4c9e-8ad4-1825595f2b8b None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:31.156645 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279b693-2f25-4c9e-8ad4-1825595f2b8b None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:31.156917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279b693-2f25-4c9e-8ad4-1825595f2b8b None admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:31.157572 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279b693-2f25-4c9e-8ad4-1825595f2b8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:31.159030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279b693-2f25-4c9e-8ad4-1825595f2b8b None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:31.176700 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8d0f9-aa4f-4654-ba1f-d179cb9cbaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:31.177435 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8d0f9-aa4f-4654-ba1f-d179cb9cbaf9 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:31.177783 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8d0f9-aa4f-4654-ba1f-d179cb9cbaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:31.178062 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8d0f9-aa4f-4654-ba1f-d179cb9cbaf9 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:31.178730 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8d0f9-aa4f-4654-ba1f-d179cb9cbaf9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:31.179030 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8d0f9-aa4f-4654-ba1f-d179cb9cbaf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:31.180418 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8d0f9-aa4f-4654-ba1f-d179cb9cbaf9 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:31.182886 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 402/800] 10.222.0.36 () {60 vars in 1132 bytes} [Sun Oct 30 00:36:31 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:36:31.190143 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 399/801] 10.222.0.36 () {56 vars in 1115 bytes} [Sun Oct 30 00:36:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:31.196273 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279b693-2f25-4c9e-8ad4-1825595f2b8b None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:31.297152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:31.322142 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279b693-2f25-4c9e-8ad4-1825595f2b8b None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:31.360453 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55412f-6f12-4628-89f1-5b7ea628746f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:31.361100 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55412f-6f12-4628-89f1-5b7ea628746f None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:31.361375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55412f-6f12-4628-89f1-5b7ea628746f None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:31.361644 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55412f-6f12-4628-89f1-5b7ea628746f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:31.374513 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55412f-6f12-4628-89f1-5b7ea628746f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:31.376227 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55412f-6f12-4628-89f1-5b7ea628746f None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:31.394990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-9d55412f-6f12-4628-89f1-5b7ea628746f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:31.394990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:31.394990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:31.394990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:31.394990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:31.394990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:31.394990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:31.394990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:31.394990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:31.394990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:31.394990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:31.394990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:31.394990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:36:31.394990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:36:31.394990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:36:31.394990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:36:31.396858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:31.396858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:31.396858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:31.396858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:31.396858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:31.396858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:31.396858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:31.396858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:31.396858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:31.396858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:31.396858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:31.396858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:31.396858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:31.396858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:31.396858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:31.396858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:31.396858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:36:31.398484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:36:31.398484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:31.398484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:31.398484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 403/802] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:31.406200 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8d0f9-aa4f-4654-ba1f-d179cb9cbaf9 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:31.621352 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:31.642180 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8d0f9-aa4f-4654-ba1f-d179cb9cbaf9 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:31.676334 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff5c59-2fea-4883-be40-ec24c84cabda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:31.676941 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff5c59-2fea-4883-be40-ec24c84cabda None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:31.677220 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff5c59-2fea-4883-be40-ec24c84cabda None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:31.677487 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff5c59-2fea-4883-be40-ec24c84cabda None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:31.678129 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff5c59-2fea-4883-be40-ec24c84cabda None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:31.678425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff5c59-2fea-4883-be40-ec24c84cabda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:31.679810 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff5c59-2fea-4883-be40-ec24c84cabda None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:31.694014 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 400/803] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:31.699620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55412f-6f12-4628-89f1-5b7ea628746f None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:31.817298 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55412f-6f12-4628-89f1-5b7ea628746f None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:31.847511 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:31.853983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d05cd-91da-4d73-a103-c0ddddc6b87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:31.854615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d05cd-91da-4d73-a103-c0ddddc6b87f None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:31.854897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d05cd-91da-4d73-a103-c0ddddc6b87f None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:31.855168 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d05cd-91da-4d73-a103-c0ddddc6b87f None admin] PATH_INFO: `/v3/users/nova` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:31.863559 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d05cd-91da-4d73-a103-c0ddddc6b87f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:31.865039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d05cd-91da-4d73-a103-c0ddddc6b87f None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:31.877631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-f82d05cd-91da-4d73-a103-c0ddddc6b87f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:36:31.877631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:31.877631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:31.877631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:31.877631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:31.877631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:31.877631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:31.877631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:31.877631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:31.877631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:31.877631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:31.877631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:31.877631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:36:31.877631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:36:31.877631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:36:31.877631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:36:31.879186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:31.879186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:31.879186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:36:31.879186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:36:31.879186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:36:31.879186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:36:31.879186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:31.879186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:31.879186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:31.879186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:31.879186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:31.879186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:31.879186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:31.879186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:31.879186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:31.879186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:31.879186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:31.879186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:31.880872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:31.880872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:31.880872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:36:31.880872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:36:31.880872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:36:31.880872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:36:31.880872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:36:31.880872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:36:31.880872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:36:31.880872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:31.880872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 404/804] 10.222.0.36 () {56 vars in 1093 bytes} [Sun Oct 30 00:36:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:31.884786 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff5c59-2fea-4883-be40-ec24c84cabda None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:31.990613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff5c59-2fea-4883-be40-ec24c84cabda None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:32.026671 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178e97c-d4ec-4928-8120-9da304af0c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:32.027286 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178e97c-d4ec-4928-8120-9da304af0c19 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:32.027563 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178e97c-d4ec-4928-8120-9da304af0c19 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:32.027833 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178e97c-d4ec-4928-8120-9da304af0c19 None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:32.028602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178e97c-d4ec-4928-8120-9da304af0c19 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:32.029826 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178e97c-d4ec-4928-8120-9da304af0c19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:32.031271 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178e97c-d4ec-4928-8120-9da304af0c19 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:32.051665 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 401/805] 10.222.0.36 () {56 vars in 1133 bytes} [Sun Oct 30 00:36:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:32.057527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d05cd-91da-4d73-a103-c0ddddc6b87f None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:32.168012 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d05cd-91da-4d73-a103-c0ddddc6b87f None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:32.203895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad378b42-0016-4798-8dfc-2592ba8ff6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:32.204551 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad378b42-0016-4798-8dfc-2592ba8ff6ba None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:32.204832 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad378b42-0016-4798-8dfc-2592ba8ff6ba None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:32.205104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad378b42-0016-4798-8dfc-2592ba8ff6ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:32.215895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad378b42-0016-4798-8dfc-2592ba8ff6ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:32.217439 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad378b42-0016-4798-8dfc-2592ba8ff6ba None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:32.233636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-ad378b42-0016-4798-8dfc-2592ba8ff6ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:32.233636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:32.233636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:32.233636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:32.233636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:32.233636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:32.233636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:32.233636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:32.233636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:32.233636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:32.233636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:32.233636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:32.233636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:36:32.233636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:36:32.233636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:36:32.233636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:36:32.235051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:32.235051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:32.235051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:32.235051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:32.235051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:32.235051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:32.235051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:32.235051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:32.235051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:32.235051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:32.235051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:32.235051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:32.235051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:32.235051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:32.235051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:32.235051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:32.235051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:36:32.236014 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:36:32.236014 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:32.236014 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:32.236014 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 405/806] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:32.241778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178e97c-d4ec-4928-8120-9da304af0c19 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:32.245772 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:32.345468 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178e97c-d4ec-4928-8120-9da304af0c19 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:32.386043 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07aaee9-7c54-4a16-a87c-3b993b390415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:32.386673 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07aaee9-7c54-4a16-a87c-3b993b390415 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:32.386951 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07aaee9-7c54-4a16-a87c-3b993b390415 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:32.387221 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07aaee9-7c54-4a16-a87c-3b993b390415 None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:32.387986 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07aaee9-7c54-4a16-a87c-3b993b390415 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:32.389067 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07aaee9-7c54-4a16-a87c-3b993b390415 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:32.391021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07aaee9-7c54-4a16-a87c-3b993b390415 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:32.403065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 402/807] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:32.409441 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad378b42-0016-4798-8dfc-2592ba8ff6ba None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:32.463336 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:32.535145 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad378b42-0016-4798-8dfc-2592ba8ff6ba None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:32.571379 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678e6fa-fe48-4705-a3b9-fba7018a389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:32.572018 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678e6fa-fe48-4705-a3b9-fba7018a389b None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:32.572327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678e6fa-fe48-4705-a3b9-fba7018a389b None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:32.572605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678e6fa-fe48-4705-a3b9-fba7018a389b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:32.579897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678e6fa-fe48-4705-a3b9-fba7018a389b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:32.581495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678e6fa-fe48-4705-a3b9-fba7018a389b None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:32.593685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-0678e6fa-fe48-4705-a3b9-fba7018a389b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:36:32.593685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:32.593685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:32.593685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:32.593685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:32.593685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:32.593685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:32.593685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:32.593685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:32.593685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:32.593685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:32.593685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:32.593685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:36:32.593685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:36:32.593685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:36:32.594643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:36:32.594643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:32.594643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:32.594643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:32.594643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:32.594643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:32.594643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:32.594643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:32.594643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:32.594643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:32.594643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:32.594643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:32.594643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:32.594643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:32.594643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:32.594643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:32.594643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:32.595544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:36:32.595544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:36:32.595544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:36:32.595544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:36:32.595544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:36:32.595544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:36:32.595544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:36:32.595544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:32.599760 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 406/808] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:36:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:32.601075 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07aaee9-7c54-4a16-a87c-3b993b390415 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:32.668315 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:32.711678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07aaee9-7c54-4a16-a87c-3b993b390415 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:32.750856 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae815b4-855d-4062-b63d-edae70b0c2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:32.751466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae815b4-855d-4062-b63d-edae70b0c2c2 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:32.751742 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae815b4-855d-4062-b63d-edae70b0c2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:32.752012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae815b4-855d-4062-b63d-edae70b0c2c2 None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:32.752732 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae815b4-855d-4062-b63d-edae70b0c2c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:32.753033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae815b4-855d-4062-b63d-edae70b0c2c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:32.754473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae815b4-855d-4062-b63d-edae70b0c2c2 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:32.768612 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 403/809] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:36:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:32.774757 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678e6fa-fe48-4705-a3b9-fba7018a389b None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:32.883135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678e6fa-fe48-4705-a3b9-fba7018a389b None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:32.886399 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:32.922459 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989914d-7fb3-4eae-8347-41b300c6d108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:32.923128 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989914d-7fb3-4eae-8347-41b300c6d108 None admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:32.923408 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989914d-7fb3-4eae-8347-41b300c6d108 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:32.923693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989914d-7fb3-4eae-8347-41b300c6d108 None admin] PATH_INFO: `/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/adbba7be39a64230bab160ce77ce7586/roles/f74a886e30fd49cd82e0a6ed7e0f7927` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:32.925863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989914d-7fb3-4eae-8347-41b300c6d108 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab3a0e5fc5a344739b58ec8313639496, user_id=adbba7be39a64230bab160ce77ce7586, role_id=f74a886e30fd49cd82e0a6ed7e0f7927)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:32.927331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989914d-7fb3-4eae-8347-41b300c6d108 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:32.941117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 407/810] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:36:32 2022] PUT /identity/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/adbba7be39a64230bab160ce77ce7586/roles/f74a886e30fd49cd82e0a6ed7e0f7927 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:36:33.494128 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e0510-0aab-4925-80bb-9a9ed2699446 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:33.494405 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e0510-0aab-4925-80bb-9a9ed2699446 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:33.494680 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e0510-0aab-4925-80bb-9a9ed2699446 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:33.496035 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 404/811] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:33.502506 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf00ad-cbd6-4c09-97f0-764d422594f3 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:33.502795 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf00ad-cbd6-4c09-97f0-764d422594f3 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:33.503074 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf00ad-cbd6-4c09-97f0-764d422594f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:33.521423 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-14bf00ad-cbd6-4c09-97f0-764d422594f3 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:33.522637 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-14bf00ad-cbd6-4c09-97f0-764d422594f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:33.593761 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 408/812] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:33.869258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e0510-0aab-4925-80bb-9a9ed2699446 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:33.985273 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e0510-0aab-4925-80bb-9a9ed2699446 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:34.008974 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:34.020944 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bec4b9-e52f-4b83-8d81-e3fda589f7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:34.021242 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bec4b9-e52f-4b83-8d81-e3fda589f7aa None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:34.021738 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bec4b9-e52f-4b83-8d81-e3fda589f7aa None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:34.021738 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bec4b9-e52f-4b83-8d81-e3fda589f7aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:34.033636 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bec4b9-e52f-4b83-8d81-e3fda589f7aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:36:34.038466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-57bec4b9-e52f-4b83-8d81-e3fda589f7aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:36:34.038466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:34.038466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:34.038466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:34.038466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:34.038466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:34.038466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:34.038466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:34.038466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:34.038466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:34.038466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:34.038466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:34.038466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:36:34.038466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:36:34.038466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:36:34.038466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:36:34.039547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:36:34.039547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:36:34.039547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:36:34.039547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:36:34.039547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:34.039547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:34.039547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:34.039547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:34.039547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:34.039547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:34.039547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:34.039547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:34.039547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:34.039547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:34.039547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:34.039547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:34.040426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:34.040426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:34.040426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:34.040426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:34.040426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:36:34.040426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:36:34.040426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:36:34.040426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:36:34.040426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:36:34.040426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:36:34.040426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:36:34.040426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:34.040426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 405/813] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:36:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:34.046627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf00ad-cbd6-4c09-97f0-764d422594f3 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:34.147774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:34.161895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf00ad-cbd6-4c09-97f0-764d422594f3 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:34.197498 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8affc-a384-4d76-b165-7ea8ecf68d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:34.198103 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8affc-a384-4d76-b165-7ea8ecf68d2f None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:34.198379 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8affc-a384-4d76-b165-7ea8ecf68d2f None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:34.198648 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8affc-a384-4d76-b165-7ea8ecf68d2f None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:34.199306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8affc-a384-4d76-b165-7ea8ecf68d2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:34.199603 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8affc-a384-4d76-b165-7ea8ecf68d2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:34.201506 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8affc-a384-4d76-b165-7ea8ecf68d2f None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:34.210499 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 409/814] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:36:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:34.216441 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bec4b9-e52f-4b83-8d81-e3fda589f7aa None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:34.326369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bec4b9-e52f-4b83-8d81-e3fda589f7aa None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:34.360409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce22320-978a-4d0c-be95-f26dbafb1e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:34.361043 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce22320-978a-4d0c-be95-f26dbafb1e51 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:34.361318 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce22320-978a-4d0c-be95-f26dbafb1e51 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:34.361591 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce22320-978a-4d0c-be95-f26dbafb1e51 None admin] PATH_INFO: `/v3/users/c66bd885e10f436a9c19a44174e90c36` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:34.375563 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce22320-978a-4d0c-be95-f26dbafb1e51 None admin] RBAC: Authorizing `identity:get_user(user_id=c66bd885e10f436a9c19a44174e90c36)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:34.377024 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce22320-978a-4d0c-be95-f26dbafb1e51 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:34.383321 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 406/815] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:36:34 2022] GET /identity/v3/users/c66bd885e10f436a9c19a44174e90c36 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:34.384752 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8affc-a384-4d76-b165-7ea8ecf68d2f None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:34.493142 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8affc-a384-4d76-b165-7ea8ecf68d2f None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:34.527686 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb0428-8769-45fa-b8d8-4a485eba63e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:34.528348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb0428-8769-45fa-b8d8-4a485eba63e2 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:34.528624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb0428-8769-45fa-b8d8-4a485eba63e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:34.528923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb0428-8769-45fa-b8d8-4a485eba63e2 None admin] PATH_INFO: `/v3/projects/67697e6849c5470e9b8386659be18147` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:34.530045 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb0428-8769-45fa-b8d8-4a485eba63e2 None admin] RBAC: Authorizing `identity:get_project(project_id=67697e6849c5470e9b8386659be18147)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:34.531485 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb0428-8769-45fa-b8d8-4a485eba63e2 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:34.533160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 410/816] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:36:34 2022] GET /identity/v3/projects/67697e6849c5470e9b8386659be18147 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:34.539090 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce22320-978a-4d0c-be95-f26dbafb1e51 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:34.583547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:34.643384 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce22320-978a-4d0c-be95-f26dbafb1e51 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:34.677971 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77067a9-1203-4805-8184-162a9b8daa92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:34.678636 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77067a9-1203-4805-8184-162a9b8daa92 None admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:34.678915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77067a9-1203-4805-8184-162a9b8daa92 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:34.679189 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77067a9-1203-4805-8184-162a9b8daa92 None admin] PATH_INFO: `/v3/projects/67697e6849c5470e9b8386659be18147/users/c66bd885e10f436a9c19a44174e90c36/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:34.681094 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77067a9-1203-4805-8184-162a9b8daa92 None admin] RBAC: Authorizing `identity:create_grant(project_id=67697e6849c5470e9b8386659be18147, user_id=c66bd885e10f436a9c19a44174e90c36, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:34.682530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77067a9-1203-4805-8184-162a9b8daa92 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:34.696269 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 407/817] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:36:34 2022] PUT /identity/v3/projects/67697e6849c5470e9b8386659be18147/users/c66bd885e10f436a9c19a44174e90c36/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:36:35.186296 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1147d-7d9f-492a-8a92-6a49a9b4958d None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:35.186850 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1147d-7d9f-492a-8a92-6a49a9b4958d None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:35.186850 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1147d-7d9f-492a-8a92-6a49a9b4958d None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:35.188213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 411/818] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:35.193831 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588e688-17e1-47a5-96b0-9d2adc674806 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:35.194317 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588e688-17e1-47a5-96b0-9d2adc674806 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:35.194639 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588e688-17e1-47a5-96b0-9d2adc674806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:35.214060 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-0588e688-17e1-47a5-96b0-9d2adc674806 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:35.215268 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-0588e688-17e1-47a5-96b0-9d2adc674806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:35.290684 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 408/819] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:35.561378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1147d-7d9f-492a-8a92-6a49a9b4958d None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:35.588230 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:35.673352 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1147d-7d9f-492a-8a92-6a49a9b4958d None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:35.710970 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3f474-0b9e-4955-91f3-68d61629d1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:35.711595 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3f474-0b9e-4955-91f3-68d61629d1af None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:35.711874 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3f474-0b9e-4955-91f3-68d61629d1af None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:35.712170 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3f474-0b9e-4955-91f3-68d61629d1af None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:35.724407 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3f474-0b9e-4955-91f3-68d61629d1af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:36:35.729327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-72e3f474-0b9e-4955-91f3-68d61629d1af None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 30 00:36:35.729327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:35.729327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:35.729327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:35.729327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:35.729327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:35.729327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:35.729327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:35.729327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:35.729327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:35.729327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:35.729327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:35.729327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:36:35.729327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:36:35.729327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:36:35.729327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:36:35.730295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:36:35.730295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:36:35.730295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:36:35.730295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:36:35.730295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:35.730295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:35.730295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:35.730295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:35.730295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:35.730295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:35.730295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:35.730295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:35.730295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:35.730295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:35.730295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:35.730295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:35.731192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:35.731192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:35.731192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:35.731192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:35.731192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:36:35.731192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:36:35.731192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:36:35.731192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:36:35.731192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:36:35.731192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:36:35.731192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 30 00:36:35.731192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:35.731192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 412/820] 10.222.0.36 () {56 vars in 1120 bytes} [Sun Oct 30 00:36:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:35.737002 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588e688-17e1-47a5-96b0-9d2adc674806 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:35.770974 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:35.845135 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588e688-17e1-47a5-96b0-9d2adc674806 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:35.881993 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a4784-d1f1-405f-9a34-255aef7ad840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:35.882609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a4784-d1f1-405f-9a34-255aef7ad840 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:35.882886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a4784-d1f1-405f-9a34-255aef7ad840 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:35.883157 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a4784-d1f1-405f-9a34-255aef7ad840 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:35.883820 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a4784-d1f1-405f-9a34-255aef7ad840 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:35.884147 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a4784-d1f1-405f-9a34-255aef7ad840 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:35.885514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a4784-d1f1-405f-9a34-255aef7ad840 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:35.900542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 409/821] 10.222.0.36 () {56 vars in 1115 bytes} [Sun Oct 30 00:36:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:35.902109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3f474-0b9e-4955-91f3-68d61629d1af None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:35.989495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:36.009825 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3f474-0b9e-4955-91f3-68d61629d1af None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:36.046991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab90543-228c-49bb-bbff-25c9a40e7ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:36.047623 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab90543-228c-49bb-bbff-25c9a40e7ce0 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:36.047906 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab90543-228c-49bb-bbff-25c9a40e7ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:36.048222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab90543-228c-49bb-bbff-25c9a40e7ce0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:36.058706 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab90543-228c-49bb-bbff-25c9a40e7ce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:36.060220 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab90543-228c-49bb-bbff-25c9a40e7ce0 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:36.076703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-5ab90543-228c-49bb-bbff-25c9a40e7ce0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:36.076703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:36.076703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:36.076703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:36.076703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:36.076703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:36.076703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:36.076703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:36.076703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:36.076703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:36.076703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:36.076703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:36.076703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:36:36.076703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:36:36.076703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:36:36.076703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:36:36.077732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:36.077732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:36.077732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:36.077732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:36.077732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:36.077732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:36.077732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:36.077732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:36.077732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:36.077732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:36.077732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:36.077732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:36.077732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:36.077732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:36.077732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:36.077732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:36.077732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:36:36.078638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:36:36.078638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:36.078638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:36.078638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 413/822] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:36.083837 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a4784-d1f1-405f-9a34-255aef7ad840 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:36.185198 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:36.193131 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a4784-d1f1-405f-9a34-255aef7ad840 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:36.237779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac0edd-2aa6-48e7-8fa8-6bae502e94fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:36.238451 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac0edd-2aa6-48e7-8fa8-6bae502e94fe None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:36.238661 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac0edd-2aa6-48e7-8fa8-6bae502e94fe None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:36.238932 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac0edd-2aa6-48e7-8fa8-6bae502e94fe None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:36.239574 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac0edd-2aa6-48e7-8fa8-6bae502e94fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:36.239877 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac0edd-2aa6-48e7-8fa8-6bae502e94fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:36.241344 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac0edd-2aa6-48e7-8fa8-6bae502e94fe None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:36.253553 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 410/823] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:36.259346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab90543-228c-49bb-bbff-25c9a40e7ce0 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:36.364814 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab90543-228c-49bb-bbff-25c9a40e7ce0 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:36.399899 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed371b0-66fa-49ee-a8f4-2f165cf58e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:36.400654 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed371b0-66fa-49ee-a8f4-2f165cf58e17 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:36.400933 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed371b0-66fa-49ee-a8f4-2f165cf58e17 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:36.401218 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed371b0-66fa-49ee-a8f4-2f165cf58e17 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:36.403242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:36.410621 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed371b0-66fa-49ee-a8f4-2f165cf58e17 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:36.412411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed371b0-66fa-49ee-a8f4-2f165cf58e17 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:36.431863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-7ed371b0-66fa-49ee-a8f4-2f165cf58e17 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:36:36.431863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:36.431863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:36.431863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:36.431863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:36.431863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:36.431863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:36.431863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:36.431863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:36.431863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:36.431863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:36.431863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:36.431863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:36:36.431863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:36:36.431863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:36:36.431863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:36:36.432917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:36.432917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:36.432917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:36:36.432917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:36:36.432917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:36:36.432917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:36:36.432917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:36.432917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:36.432917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:36.432917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:36.432917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:36.432917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:36.432917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:36.432917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:36.432917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:36.432917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:36.432917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:36.432917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:36.433873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:36.433873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:36.433873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:36:36.433873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:36:36.433873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:36:36.433873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:36:36.433873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:36:36.433873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:36:36.433873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 30 00:36:36.433873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:36.433873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 414/824] 10.222.0.36 () {56 vars in 1093 bytes} [Sun Oct 30 00:36:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:36.440204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac0edd-2aa6-48e7-8fa8-6bae502e94fe None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:36.543299 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac0edd-2aa6-48e7-8fa8-6bae502e94fe None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:36.579212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abd668-8c6f-4c90-bb33-15c179a3144b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:36.579826 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abd668-8c6f-4c90-bb33-15c179a3144b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:36.580123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abd668-8c6f-4c90-bb33-15c179a3144b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:36.580397 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abd668-8c6f-4c90-bb33-15c179a3144b None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:36.582264 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abd668-8c6f-4c90-bb33-15c179a3144b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:36.582264 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abd668-8c6f-4c90-bb33-15c179a3144b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:36.582871 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abd668-8c6f-4c90-bb33-15c179a3144b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:36.600454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 411/825] 10.222.0.36 () {56 vars in 1133 bytes} [Sun Oct 30 00:36:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:36.606134 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed371b0-66fa-49ee-a8f4-2f165cf58e17 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:36.714742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed371b0-66fa-49ee-a8f4-2f165cf58e17 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:36.752745 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d45cfe-f875-4b09-94bc-8c2a3ec72ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:36.753359 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d45cfe-f875-4b09-94bc-8c2a3ec72ab7 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:36.753639 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d45cfe-f875-4b09-94bc-8c2a3ec72ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:36.753914 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d45cfe-f875-4b09-94bc-8c2a3ec72ab7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:36.823925 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:36.832695 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d45cfe-f875-4b09-94bc-8c2a3ec72ab7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:36.834185 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d45cfe-f875-4b09-94bc-8c2a3ec72ab7 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:36.897675 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c6230-f1a0-43ee-a702-b8b11b4967a4 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:36.897960 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c6230-f1a0-43ee-a702-b8b11b4967a4 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:36.898224 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c6230-f1a0-43ee-a702-b8b11b4967a4 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:36.899565 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 412/826] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:36.905939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a734588-c0fe-4bba-9ca6-884135ef8549 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:36.906217 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a734588-c0fe-4bba-9ca6-884135ef8549 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:36.906498 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a734588-c0fe-4bba-9ca6-884135ef8549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:36.925516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-3a734588-c0fe-4bba-9ca6-884135ef8549 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:36.926720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-3a734588-c0fe-4bba-9ca6-884135ef8549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:36.934674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:36.944166 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 415/827] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:36:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 30 00:36:36.950260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d45cfe-f875-4b09-94bc-8c2a3ec72ab7 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:37.004226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 413/828] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:36:37.049358 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:37.058877 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d45cfe-f875-4b09-94bc-8c2a3ec72ab7 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:37.094093 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ec3d2-81e3-4088-951f-62cfa6323840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:37.094724 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ec3d2-81e3-4088-951f-62cfa6323840 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:37.095005 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ec3d2-81e3-4088-951f-62cfa6323840 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:37.095277 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ec3d2-81e3-4088-951f-62cfa6323840 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:37.105570 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ec3d2-81e3-4088-951f-62cfa6323840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:37.107050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ec3d2-81e3-4088-951f-62cfa6323840 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:37.124367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-335ec3d2-81e3-4088-951f-62cfa6323840 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:37.124367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:37.124367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:37.124367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:37.124367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:37.124367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:37.124367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:37.124367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:37.124367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:37.124367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:37.124367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:37.124367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:37.124367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:36:37.124367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:36:37.124367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:36:37.124367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:36:37.125426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:37.125426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:37.125426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:37.125426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:37.125426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:37.125426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:37.125426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:37.125426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:37.125426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:37.125426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:37.125426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:37.125426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:37.125426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:37.125426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:37.125426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:37.125426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:37.125426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:36:37.126334 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:36:37.126334 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:36:37.126334 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:37.126334 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 416/829] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:36:37.131478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a734588-c0fe-4bba-9ca6-884135ef8549 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:37.242711 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a734588-c0fe-4bba-9ca6-884135ef8549 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:37.276613 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ec3d2-81e3-4088-951f-62cfa6323840 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:37.280502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77ad79-8f9c-47d0-84a3-6d3e30ea190d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:37.281134 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77ad79-8f9c-47d0-84a3-6d3e30ea190d None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:37.281411 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77ad79-8f9c-47d0-84a3-6d3e30ea190d None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:37.281683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77ad79-8f9c-47d0-84a3-6d3e30ea190d None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:37.282331 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77ad79-8f9c-47d0-84a3-6d3e30ea190d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:37.282633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77ad79-8f9c-47d0-84a3-6d3e30ea190d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:37.284017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77ad79-8f9c-47d0-84a3-6d3e30ea190d None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:37.296467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 414/830] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:37.302265 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77ad79-8f9c-47d0-84a3-6d3e30ea190d None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:37.305101 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:37.387894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ec3d2-81e3-4088-951f-62cfa6323840 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:37.416559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77ad79-8f9c-47d0-84a3-6d3e30ea190d None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:37.424055 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cf88d-2856-45cb-9772-dfa28aa7e341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:37.424717 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cf88d-2856-45cb-9772-dfa28aa7e341 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:37.425007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cf88d-2856-45cb-9772-dfa28aa7e341 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:37.425281 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cf88d-2856-45cb-9772-dfa28aa7e341 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:37.436216 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40cf88d-2856-45cb-9772-dfa28aa7e341 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:36:37.441636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-b40cf88d-2856-45cb-9772-dfa28aa7e341 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:36:37.441636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:37.441636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:37.441636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:37.441636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:37.441636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:37.441636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:37.441636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:37.441636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:37.441636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:37.441636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:37.441636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:37.441636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:36:37.441636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:36:37.441636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:36:37.441636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:36:37.442955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:36:37.442955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:36:37.442955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:36:37.442955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:36:37.442955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:37.442955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:37.442955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:37.442955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:37.442955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:37.442955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:37.442955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:37.442955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:37.442955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:37.442955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:37.442955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:37.442955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:37.444609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:37.444609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:37.444609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:37.444609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:37.444609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:36:37.444609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:36:37.444609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:36:37.444609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:36:37.444609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:36:37.444609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:36:37.444609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:36:37.444609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:37.444609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 417/831] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:36:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:36:37.448900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cf88d-2856-45cb-9772-dfa28aa7e341 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:37.455528 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c802ad5-a772-4d39-859a-27d811b0324b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:37.456206 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c802ad5-a772-4d39-859a-27d811b0324b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:37.456487 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c802ad5-a772-4d39-859a-27d811b0324b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:37.456760 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c802ad5-a772-4d39-859a-27d811b0324b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:37.458340 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:37.464667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c802ad5-a772-4d39-859a-27d811b0324b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:37.466187 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c802ad5-a772-4d39-859a-27d811b0324b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:37.479073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-5c802ad5-a772-4d39-859a-27d811b0324b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:36:37.479073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:37.479073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:37.479073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:37.479073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:37.479073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:37.479073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:37.479073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:37.479073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:37.479073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:37.479073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:37.479073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:37.479073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:36:37.479073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:36:37.479073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:36:37.480146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:36:37.480146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:37.480146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:37.480146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:37.480146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:37.480146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:37.480146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:37.480146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:37.480146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:37.480146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:37.480146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:37.480146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:37.480146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:37.480146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:37.480146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:37.480146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:37.480146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:37.481258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:36:37.481258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:36:37.481258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:36:37.481258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:36:37.481258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:36:37.481258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:36:37.481258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:36:37.481258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:36:37.481258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 415/832] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:36:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:36:37.487240 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c802ad5-a772-4d39-859a-27d811b0324b None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:37.552912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:37.562888 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cf88d-2856-45cb-9772-dfa28aa7e341 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:37.594537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c802ad5-a772-4d39-859a-27d811b0324b None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:37.599936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868fa94-529a-484e-94ce-b8be843e27f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:37.599936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868fa94-529a-484e-94ce-b8be843e27f4 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:37.600233 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868fa94-529a-484e-94ce-b8be843e27f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:37.600349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868fa94-529a-484e-94ce-b8be843e27f4 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:37.600991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868fa94-529a-484e-94ce-b8be843e27f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:37.601293 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868fa94-529a-484e-94ce-b8be843e27f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:37.602705 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868fa94-529a-484e-94ce-b8be843e27f4 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:37.612783 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 418/833] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:36:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:37.618777 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868fa94-529a-484e-94ce-b8be843e27f4 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:37.629093 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383344a-8a26-4a3e-a202-46d7cbe4260f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:37.629732 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383344a-8a26-4a3e-a202-46d7cbe4260f None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:37.629990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383344a-8a26-4a3e-a202-46d7cbe4260f None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:37.630261 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383344a-8a26-4a3e-a202-46d7cbe4260f None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:37.630954 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383344a-8a26-4a3e-a202-46d7cbe4260f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:37.631258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383344a-8a26-4a3e-a202-46d7cbe4260f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:37.632740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383344a-8a26-4a3e-a202-46d7cbe4260f None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:37.656138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 416/834] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:36:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:37.657333 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383344a-8a26-4a3e-a202-46d7cbe4260f None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:37.690765 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:37.727752 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868fa94-529a-484e-94ce-b8be843e27f4 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:37.760991 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383344a-8a26-4a3e-a202-46d7cbe4260f None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:37.767817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966df17-7881-459e-ac49-ff933b401777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:37.768493 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966df17-7881-459e-ac49-ff933b401777 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:37.768803 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966df17-7881-459e-ac49-ff933b401777 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:37.769558 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966df17-7881-459e-ac49-ff933b401777 None admin] PATH_INFO: `/v3/users/c66bd885e10f436a9c19a44174e90c36` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:37.770714 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966df17-7881-459e-ac49-ff933b401777 None admin] RBAC: Authorizing `identity:get_user(user_id=c66bd885e10f436a9c19a44174e90c36)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:37.773016 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966df17-7881-459e-ac49-ff933b401777 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:37.774655 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 419/835] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:36:37 2022] GET /identity/v3/users/c66bd885e10f436a9c19a44174e90c36 => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:37.781011 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966df17-7881-459e-ac49-ff933b401777 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:37.786568 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:37.805013 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64412b42-0c68-4ce6-8ac7-0bd49b57a80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:37.805547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64412b42-0c68-4ce6-8ac7-0bd49b57a80d None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:37.805835 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64412b42-0c68-4ce6-8ac7-0bd49b57a80d None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:37.806107 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64412b42-0c68-4ce6-8ac7-0bd49b57a80d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:37.806837 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64412b42-0c68-4ce6-8ac7-0bd49b57a80d None admin] RBAC: Adding query filter params (role.id=f74a886e30fd49cd82e0a6ed7e0f7927, scope.project.id=ab3a0e5fc5a344739b58ec8313639496, user.id=adbba7be39a64230bab160ce77ce7586) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:37.807144 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64412b42-0c68-4ce6-8ac7-0bd49b57a80d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:37.808648 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64412b42-0c68-4ce6-8ac7-0bd49b57a80d None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:37.814842 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 417/836] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:36:37 2022] GET /identity/v3/role_assignments?user.id=adbba7be39a64230bab160ce77ce7586&scope.project.id=ab3a0e5fc5a344739b58ec8313639496&role.id=f74a886e30fd49cd82e0a6ed7e0f7927 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:37.902181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966df17-7881-459e-ac49-ff933b401777 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:37.940600 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382306f-bfcc-4a22-a7c2-516236e72946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:37.941229 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382306f-bfcc-4a22-a7c2-516236e72946 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:37.941506 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382306f-bfcc-4a22-a7c2-516236e72946 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:37.941778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382306f-bfcc-4a22-a7c2-516236e72946 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:37.945876 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:38.018923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382306f-bfcc-4a22-a7c2-516236e72946 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:38.020416 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382306f-bfcc-4a22-a7c2-516236e72946 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:38.130643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 420/837] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:36:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 30 00:36:38.139040 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64412b42-0c68-4ce6-8ac7-0bd49b57a80d None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:38.256277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64412b42-0c68-4ce6-8ac7-0bd49b57a80d None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:38.291476 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64604d80-a673-43c0-aff4-79273e1d34ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:38.292137 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64604d80-a673-43c0-aff4-79273e1d34ee None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:38.292417 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64604d80-a673-43c0-aff4-79273e1d34ee None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:38.292690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64604d80-a673-43c0-aff4-79273e1d34ee None admin] PATH_INFO: `/v3/projects/67697e6849c5470e9b8386659be18147` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:38.293829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64604d80-a673-43c0-aff4-79273e1d34ee None admin] RBAC: Authorizing `identity:get_project(project_id=67697e6849c5470e9b8386659be18147)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:38.295616 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64604d80-a673-43c0-aff4-79273e1d34ee None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:38.297457 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 418/838] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:36:38 2022] GET /identity/v3/projects/67697e6849c5470e9b8386659be18147 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:38.303726 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382306f-bfcc-4a22-a7c2-516236e72946 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:38.341697 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:38.413228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382306f-bfcc-4a22-a7c2-516236e72946 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:38.450242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed6c4b-ce7c-41b1-a25d-a49f46367dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:38.450849 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed6c4b-ce7c-41b1-a25d-a49f46367dc4 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:38.451181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed6c4b-ce7c-41b1-a25d-a49f46367dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:38.451941 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed6c4b-ce7c-41b1-a25d-a49f46367dc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:38.452700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed6c4b-ce7c-41b1-a25d-a49f46367dc4 None admin] RBAC: Adding query filter params (role.id=980241d45fdb435d9f32c434057c71e4, scope.project.id=67697e6849c5470e9b8386659be18147, user.id=c66bd885e10f436a9c19a44174e90c36) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:38.453004 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed6c4b-ce7c-41b1-a25d-a49f46367dc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:38.454459 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed6c4b-ce7c-41b1-a25d-a49f46367dc4 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:38.460773 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 421/839] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:36:38 2022] GET /identity/v3/role_assignments?user.id=c66bd885e10f436a9c19a44174e90c36&scope.project.id=67697e6849c5470e9b8386659be18147&role.id=980241d45fdb435d9f32c434057c71e4 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:36:40.663937 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d86fb-7edf-4d1f-ae67-e447ef20bc51 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:40.664519 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d86fb-7edf-4d1f-ae67-e447ef20bc51 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:40.664519 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d86fb-7edf-4d1f-ae67-e447ef20bc51 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:40.665860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 419/840] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:40.672516 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb538bb-2ab8-4192-9396-74e354bc6e66 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:40.672983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb538bb-2ab8-4192-9396-74e354bc6e66 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:40.673073 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb538bb-2ab8-4192-9396-74e354bc6e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:40.691798 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-0bb538bb-2ab8-4192-9396-74e354bc6e66 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:40.693058 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-0bb538bb-2ab8-4192-9396-74e354bc6e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:40.759122 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 422/841] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:41.034118 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d86fb-7edf-4d1f-ae67-e447ef20bc51 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:41.092658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:41.147640 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d86fb-7edf-4d1f-ae67-e447ef20bc51 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:41.193171 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a218099e-1546-4144-aee0-485d6848d0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:41.194200 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218099e-1546-4144-aee0-485d6848d0c8 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:41.194699 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218099e-1546-4144-aee0-485d6848d0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:41.195148 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a218099e-1546-4144-aee0-485d6848d0c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:41.196988 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a218099e-1546-4144-aee0-485d6848d0c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:41.199766 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a218099e-1546-4144-aee0-485d6848d0c8 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:41.204520 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 420/842] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:41.210012 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb538bb-2ab8-4192-9396-74e354bc6e66 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:41.243664 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:41.328638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb538bb-2ab8-4192-9396-74e354bc6e66 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:41.372890 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba651765-dee6-452c-8876-d90ff1d6e327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:41.373516 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba651765-dee6-452c-8876-d90ff1d6e327 None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:41.373792 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba651765-dee6-452c-8876-d90ff1d6e327 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:41.374065 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba651765-dee6-452c-8876-d90ff1d6e327 None admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:41.374717 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba651765-dee6-452c-8876-d90ff1d6e327 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:41.377013 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba651765-dee6-452c-8876-d90ff1d6e327 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:41.401221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 423/843] 10.222.0.36 () {60 vars in 1132 bytes} [Sun Oct 30 00:36:41 2022] POST /identity/v3/users => generated 286 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:36:43.642500 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42ff4f-35a9-44a9-a932-4b759661fa25 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:43.643037 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42ff4f-35a9-44a9-a932-4b759661fa25 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:43.643037 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42ff4f-35a9-44a9-a932-4b759661fa25 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:43.644429 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 421/844] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:43.649879 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24075dae-8d7d-4683-a3fd-7b165200fa02 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:43.650326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24075dae-8d7d-4683-a3fd-7b165200fa02 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:43.650418 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24075dae-8d7d-4683-a3fd-7b165200fa02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:43.669119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-24075dae-8d7d-4683-a3fd-7b165200fa02 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:43.670313 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-24075dae-8d7d-4683-a3fd-7b165200fa02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:43.739019 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 424/845] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:44.017874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42ff4f-35a9-44a9-a932-4b759661fa25 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:44.127467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42ff4f-35a9-44a9-a932-4b759661fa25 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:44.151917 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:44.164245 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815bbf9-cb3b-491b-9d1f-e9d1bece5253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:44.164874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815bbf9-cb3b-491b-9d1f-e9d1bece5253 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:44.165147 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815bbf9-cb3b-491b-9d1f-e9d1bece5253 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:44.165416 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815bbf9-cb3b-491b-9d1f-e9d1bece5253 None admin] PATH_INFO: `/v3/roles/af3529d63bae4650be7b8dc67b3ed527` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:44.166786 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815bbf9-cb3b-491b-9d1f-e9d1bece5253 None admin] RBAC: Authorizing `identity:get_role(role_id=af3529d63bae4650be7b8dc67b3ed527)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:44.168168 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815bbf9-cb3b-491b-9d1f-e9d1bece5253 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:44.169494 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 422/846] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:36:44 2022] GET /identity/v3/roles/af3529d63bae4650be7b8dc67b3ed527 => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:44.176784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24075dae-8d7d-4683-a3fd-7b165200fa02 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:44.194776 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:44.277951 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24075dae-8d7d-4683-a3fd-7b165200fa02 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:44.313758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96492ea-309a-481e-89fc-9bbbcc308902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:44.314395 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96492ea-309a-481e-89fc-9bbbcc308902 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:44.314679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96492ea-309a-481e-89fc-9bbbcc308902 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:44.314953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96492ea-309a-481e-89fc-9bbbcc308902 None admin] PATH_INFO: `/v3/users/d9680ed86ee84115915c5e028a8122f9` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:44.328639 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96492ea-309a-481e-89fc-9bbbcc308902 None admin] RBAC: Authorizing `identity:get_user(user_id=d9680ed86ee84115915c5e028a8122f9)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:44.330114 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96492ea-309a-481e-89fc-9bbbcc308902 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:44.331783 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 425/847] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:36:44 2022] GET /identity/v3/users/d9680ed86ee84115915c5e028a8122f9 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:44.337918 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815bbf9-cb3b-491b-9d1f-e9d1bece5253 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:44.441179 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815bbf9-cb3b-491b-9d1f-e9d1bece5253 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:44.475985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a95682-b160-4a03-bbd6-84c25339eb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:44.476659 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a95682-b160-4a03-bbd6-84c25339eb24 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:44.476933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a95682-b160-4a03-bbd6-84c25339eb24 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:44.477206 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a95682-b160-4a03-bbd6-84c25339eb24 None admin] PATH_INFO: `/v3/projects/67697e6849c5470e9b8386659be18147` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:44.478322 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a95682-b160-4a03-bbd6-84c25339eb24 None admin] RBAC: Authorizing `identity:get_project(project_id=67697e6849c5470e9b8386659be18147)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:44.479759 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a95682-b160-4a03-bbd6-84c25339eb24 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:44.481425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 423/848] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:36:44 2022] GET /identity/v3/projects/67697e6849c5470e9b8386659be18147 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:44.488375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96492ea-309a-481e-89fc-9bbbcc308902 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:44.565283 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:44.592017 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96492ea-309a-481e-89fc-9bbbcc308902 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:44.625886 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957015a-d7e2-4cfe-b690-801f2b884307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:44.626538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957015a-d7e2-4cfe-b690-801f2b884307 None admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:44.626814 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957015a-d7e2-4cfe-b690-801f2b884307 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:44.627080 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957015a-d7e2-4cfe-b690-801f2b884307 None admin] PATH_INFO: `/v3/projects/67697e6849c5470e9b8386659be18147/users/d9680ed86ee84115915c5e028a8122f9/roles/af3529d63bae4650be7b8dc67b3ed527` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:44.629210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957015a-d7e2-4cfe-b690-801f2b884307 None admin] RBAC: Authorizing `identity:create_grant(project_id=67697e6849c5470e9b8386659be18147, user_id=d9680ed86ee84115915c5e028a8122f9, role_id=af3529d63bae4650be7b8dc67b3ed527)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:44.630650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957015a-d7e2-4cfe-b690-801f2b884307 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:44.645270 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 426/849] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:36:44 2022] PUT /identity/v3/projects/67697e6849c5470e9b8386659be18147/users/d9680ed86ee84115915c5e028a8122f9/roles/af3529d63bae4650be7b8dc67b3ed527 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:36:46.838022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7c8e3-3b50-4821-9f74-6722f00a723a None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:46.838566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7c8e3-3b50-4821-9f74-6722f00a723a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:46.838566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7c8e3-3b50-4821-9f74-6722f00a723a None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:46.839885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 424/850] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:46.845468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d55667-1b07-465e-b706-feb25c2aa82e None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:46.845928 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d55667-1b07-465e-b706-feb25c2aa82e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:46.846015 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d55667-1b07-465e-b706-feb25c2aa82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:46.979969 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-54d55667-1b07-465e-b706-feb25c2aa82e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:46.981205 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-54d55667-1b07-465e-b706-feb25c2aa82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:47.050441 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 427/851] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:47.322135 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7c8e3-3b50-4821-9f74-6722f00a723a None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:47.385028 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:47.428146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7c8e3-3b50-4821-9f74-6722f00a723a None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:47.462444 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07582839-965a-429e-876d-0a50feb86c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:47.463073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07582839-965a-429e-876d-0a50feb86c03 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:47.463352 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07582839-965a-429e-876d-0a50feb86c03 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:47.463624 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07582839-965a-429e-876d-0a50feb86c03 None admin] PATH_INFO: `/v3/roles/af3529d63bae4650be7b8dc67b3ed527` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:47.465033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07582839-965a-429e-876d-0a50feb86c03 None admin] RBAC: Authorizing `identity:get_role(role_id=af3529d63bae4650be7b8dc67b3ed527)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:47.466410 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07582839-965a-429e-876d-0a50feb86c03 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:47.468311 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 425/852] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:36:47 2022] GET /identity/v3/roles/af3529d63bae4650be7b8dc67b3ed527 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:47.474194 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d55667-1b07-465e-b706-feb25c2aa82e None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:47.538250 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:47.581792 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d55667-1b07-465e-b706-feb25c2aa82e None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:47.616561 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69cfdf-de84-4c41-8810-3f91bbc54a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:47.617195 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69cfdf-de84-4c41-8810-3f91bbc54a26 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:47.617480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69cfdf-de84-4c41-8810-3f91bbc54a26 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:47.617755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69cfdf-de84-4c41-8810-3f91bbc54a26 None admin] PATH_INFO: `/v3/users/d9680ed86ee84115915c5e028a8122f9` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:47.618883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69cfdf-de84-4c41-8810-3f91bbc54a26 None admin] RBAC: Authorizing `identity:get_user(user_id=d9680ed86ee84115915c5e028a8122f9)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:47.620364 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69cfdf-de84-4c41-8810-3f91bbc54a26 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:47.621970 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 428/853] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:36:47 2022] GET /identity/v3/users/d9680ed86ee84115915c5e028a8122f9 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:47.628352 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07582839-965a-429e-876d-0a50feb86c03 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:47.730187 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07582839-965a-429e-876d-0a50feb86c03 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:47.756702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:47.767391 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aece98d-e8e5-49f5-ad09-02a0297de657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:47.768008 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aece98d-e8e5-49f5-ad09-02a0297de657 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:47.768316 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aece98d-e8e5-49f5-ad09-02a0297de657 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:47.768595 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aece98d-e8e5-49f5-ad09-02a0297de657 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:47.837347 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aece98d-e8e5-49f5-ad09-02a0297de657 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:47.838689 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aece98d-e8e5-49f5-ad09-02a0297de657 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:47.942838 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 426/854] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:36:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 30 00:36:47.949044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69cfdf-de84-4c41-8810-3f91bbc54a26 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:48.065299 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69cfdf-de84-4c41-8810-3f91bbc54a26 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:48.096623 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:48.102261 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840b17d-d108-4dd6-8fc9-f2a3ca6bdef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:48.102925 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840b17d-d108-4dd6-8fc9-f2a3ca6bdef3 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:48.103218 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840b17d-d108-4dd6-8fc9-f2a3ca6bdef3 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:48.103486 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840b17d-d108-4dd6-8fc9-f2a3ca6bdef3 None admin] PATH_INFO: `/v3/projects/67697e6849c5470e9b8386659be18147` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:48.104652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840b17d-d108-4dd6-8fc9-f2a3ca6bdef3 None admin] RBAC: Authorizing `identity:get_project(project_id=67697e6849c5470e9b8386659be18147)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:48.106093 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840b17d-d108-4dd6-8fc9-f2a3ca6bdef3 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:48.107790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 429/855] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:36:47 2022] GET /identity/v3/projects/67697e6849c5470e9b8386659be18147 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:48.114420 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aece98d-e8e5-49f5-ad09-02a0297de657 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:48.142791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:48.220981 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aece98d-e8e5-49f5-ad09-02a0297de657 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:48.255008 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd16de-ceba-4936-8968-d0da883e9cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:48.255610 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd16de-ceba-4936-8968-d0da883e9cab None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:48.255890 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd16de-ceba-4936-8968-d0da883e9cab None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:48.256187 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd16de-ceba-4936-8968-d0da883e9cab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:48.256915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd16de-ceba-4936-8968-d0da883e9cab None admin] RBAC: Adding query filter params (role.id=af3529d63bae4650be7b8dc67b3ed527, scope.project.id=67697e6849c5470e9b8386659be18147, user.id=d9680ed86ee84115915c5e028a8122f9) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:48.257215 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd16de-ceba-4936-8968-d0da883e9cab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:48.258658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd16de-ceba-4936-8968-d0da883e9cab None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:48.265371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 427/856] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:36:48 2022] GET /identity/v3/role_assignments?user.id=d9680ed86ee84115915c5e028a8122f9&scope.project.id=67697e6849c5470e9b8386659be18147&role.id=af3529d63bae4650be7b8dc67b3ed527 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:50.461954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a7932-be95-4715-8f4b-feaed45e19ba None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:50.462498 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a7932-be95-4715-8f4b-feaed45e19ba None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:50.462498 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a7932-be95-4715-8f4b-feaed45e19ba None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:50.463847 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 430/857] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:50.469903 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2d6c1-55f4-4268-b8bb-7c8a0cba0c54 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:50.470374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2d6c1-55f4-4268-b8bb-7c8a0cba0c54 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:50.470465 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2d6c1-55f4-4268-b8bb-7c8a0cba0c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:50.490188 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-55c2d6c1-55f4-4268-b8bb-7c8a0cba0c54 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:50.491498 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-55c2d6c1-55f4-4268-b8bb-7c8a0cba0c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:50.561247 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 428/858] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:50.832328 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a7932-be95-4715-8f4b-feaed45e19ba None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:50.939923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a7932-be95-4715-8f4b-feaed45e19ba None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:50.976453 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a4373-1833-464f-b0b9-76144d09f493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:50.977086 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a4373-1833-464f-b0b9-76144d09f493 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:50.977357 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a4373-1833-464f-b0b9-76144d09f493 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:50.977621 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a4373-1833-464f-b0b9-76144d09f493 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:50.978731 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a4373-1833-464f-b0b9-76144d09f493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:50.980186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322a4373-1833-464f-b0b9-76144d09f493 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:50.981753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 431/859] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:50.987691 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2d6c1-55f4-4268-b8bb-7c8a0cba0c54 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:51.009056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:51.090084 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2d6c1-55f4-4268-b8bb-7c8a0cba0c54 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:51.125266 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4583634-c4db-4358-b51f-cf02f0a58a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:51.125871 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4583634-c4db-4358-b51f-cf02f0a58a71 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:51.126149 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4583634-c4db-4358-b51f-cf02f0a58a71 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:51.126417 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4583634-c4db-4358-b51f-cf02f0a58a71 None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:51.127067 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4583634-c4db-4358-b51f-cf02f0a58a71 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:51.128526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4583634-c4db-4358-b51f-cf02f0a58a71 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:51.155331 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 429/860] 10.222.0.36 () {60 vars in 1141 bytes} [Sun Oct 30 00:36:50 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:36:53.379284 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c7ec1-2ba9-4866-807e-10a22cbd0d1e None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:53.379852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c7ec1-2ba9-4866-807e-10a22cbd0d1e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:53.379852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c7ec1-2ba9-4866-807e-10a22cbd0d1e None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:53.381179 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 432/861] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:53.386776 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2dc1f2-badd-458f-8e83-5f18bf83754c None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:53.387226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2dc1f2-badd-458f-8e83-5f18bf83754c None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:53.387317 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2dc1f2-badd-458f-8e83-5f18bf83754c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:53.413306 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-bb2dc1f2-badd-458f-8e83-5f18bf83754c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:53.414512 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-bb2dc1f2-badd-458f-8e83-5f18bf83754c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:53.485865 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 430/862] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:53.770602 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c7ec1-2ba9-4866-807e-10a22cbd0d1e None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:53.839469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:53.873908 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c7ec1-2ba9-4866-807e-10a22cbd0d1e None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:53.908981 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b26da8-fd47-4355-87ba-5d3ddfcad064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:53.909605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b26da8-fd47-4355-87ba-5d3ddfcad064 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:53.909881 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b26da8-fd47-4355-87ba-5d3ddfcad064 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:53.910151 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b26da8-fd47-4355-87ba-5d3ddfcad064 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:53.911260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b26da8-fd47-4355-87ba-5d3ddfcad064 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:53.912719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b26da8-fd47-4355-87ba-5d3ddfcad064 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:53.914287 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 433/863] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:36:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:53.920968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2dc1f2-badd-458f-8e83-5f18bf83754c None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:53.923178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:54.030839 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2dc1f2-badd-458f-8e83-5f18bf83754c None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:54.068340 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778a95f-176c-40b1-8587-43405c178567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:54.068955 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778a95f-176c-40b1-8587-43405c178567 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:54.069236 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778a95f-176c-40b1-8587-43405c178567 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:54.069510 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778a95f-176c-40b1-8587-43405c178567 None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:54.070172 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778a95f-176c-40b1-8587-43405c178567 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:54.071610 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778a95f-176c-40b1-8587-43405c178567 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:54.097625 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 431/864] 10.222.0.36 () {60 vars in 1132 bytes} [Sun Oct 30 00:36:53 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:36:56.388619 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa73cf-a5e7-4c3c-9042-c8b915f87735 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:56.389225 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa73cf-a5e7-4c3c-9042-c8b915f87735 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:56.389225 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa73cf-a5e7-4c3c-9042-c8b915f87735 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:56.390594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 434/865] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:56.396062 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2a8cf-4794-45b6-99e3-0172a2919a5f None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:56.396537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2a8cf-4794-45b6-99e3-0172a2919a5f None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:56.396631 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2a8cf-4794-45b6-99e3-0172a2919a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:56.415343 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-c2a2a8cf-4794-45b6-99e3-0172a2919a5f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:56.416557 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c2a2a8cf-4794-45b6-99e3-0172a2919a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:56.471952 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:56.487491 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 432/866] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:36:56.761970 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa73cf-a5e7-4c3c-9042-c8b915f87735 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:56.877467 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa73cf-a5e7-4c3c-9042-c8b915f87735 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:56.913338 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eaad7a-2eeb-4108-a8c3-5e11c2acb3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:56.913975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaad7a-2eeb-4108-a8c3-5e11c2acb3e2 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:56.914254 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaad7a-2eeb-4108-a8c3-5e11c2acb3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:56.914526 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaad7a-2eeb-4108-a8c3-5e11c2acb3e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:56.917224 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:56.926223 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eaad7a-2eeb-4108-a8c3-5e11c2acb3e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:36:56.931251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-a3eaad7a-2eeb-4108-a8c3-5e11c2acb3e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:36:56.931251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:36:56.931251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:36:56.931251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:36:56.931251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:36:56.931251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:36:56.931251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:36:56.931251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:36:56.931251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:36:56.931251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:36:56.931251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:36:56.931251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:36:56.931251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:36:56.931251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:36:56.931251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:36:56.931251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:36:56.932491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:36:56.932491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:36:56.932491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:36:56.932491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:36:56.932491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:36:56.932491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:36:56.932491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:36:56.932491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:36:56.932491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:36:56.932491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:36:56.932491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:36:56.932491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:36:56.932491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:36:56.932491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:36:56.932491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:36:56.932491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:36:56.933543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:36:56.933543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:36:56.933543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:36:56.933543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:36:56.933543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:36:56.933543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:36:56.933543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:36:56.933543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:36:56.933543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:36:56.933543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:36:56.933543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:36:56.933543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:36:56.936266 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 435/867] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:36:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:36:56.939078 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2a8cf-4794-45b6-99e3-0172a2919a5f None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:57.045126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2a8cf-4794-45b6-99e3-0172a2919a5f None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:57.080671 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790afef-997f-4d4a-a280-230267ba78a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:57.081330 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790afef-997f-4d4a-a280-230267ba78a3 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:57.081608 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790afef-997f-4d4a-a280-230267ba78a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:57.081882 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790afef-997f-4d4a-a280-230267ba78a3 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:57.082546 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790afef-997f-4d4a-a280-230267ba78a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:36:57.082847 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790afef-997f-4d4a-a280-230267ba78a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:57.084285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e790afef-997f-4d4a-a280-230267ba78a3 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:57.093035 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 433/868] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:36:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:57.099431 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eaad7a-2eeb-4108-a8c3-5e11c2acb3e2 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:57.209520 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eaad7a-2eeb-4108-a8c3-5e11c2acb3e2 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:57.244113 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4a899-10e9-4293-a7ca-88c4015a6a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:57.244749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4a899-10e9-4293-a7ca-88c4015a6a73 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:57.245022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4a899-10e9-4293-a7ca-88c4015a6a73 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:57.245291 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4a899-10e9-4293-a7ca-88c4015a6a73 None admin] PATH_INFO: `/v3/users/bd010554e9fe44d88abb33f109f7fbd0` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:57.265474 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4a899-10e9-4293-a7ca-88c4015a6a73 None admin] RBAC: Authorizing `identity:get_user(user_id=bd010554e9fe44d88abb33f109f7fbd0)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:57.266948 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4a899-10e9-4293-a7ca-88c4015a6a73 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:57.268716 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 436/869] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:36:57 2022] GET /identity/v3/users/bd010554e9fe44d88abb33f109f7fbd0 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:57.274531 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790afef-997f-4d4a-a280-230267ba78a3 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:57.326969 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:57.378679 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790afef-997f-4d4a-a280-230267ba78a3 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:57.411672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f1ad4-19ca-4910-bee6-ffac11379202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:57.412396 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f1ad4-19ca-4910-bee6-ffac11379202 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:57.412595 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f1ad4-19ca-4910-bee6-ffac11379202 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:57.412866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f1ad4-19ca-4910-bee6-ffac11379202 None admin] PATH_INFO: `/v3/projects/23df208b3ac7427c970d4cda0a860820` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:57.414006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f1ad4-19ca-4910-bee6-ffac11379202 None admin] RBAC: Authorizing `identity:get_project(project_id=23df208b3ac7427c970d4cda0a860820)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:57.415441 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f1ad4-19ca-4910-bee6-ffac11379202 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:57.417110 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 434/870] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:36:57 2022] GET /identity/v3/projects/23df208b3ac7427c970d4cda0a860820 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:36:57.424116 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4a899-10e9-4293-a7ca-88c4015a6a73 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:36:57.479598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:36:57.527818 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4a899-10e9-4293-a7ca-88c4015a6a73 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:36:57.568914 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ca0be-7114-4616-bfeb-4b6e5c8c6c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:36:57.569581 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ca0be-7114-4616-bfeb-4b6e5c8c6c24 None admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:57.569865 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ca0be-7114-4616-bfeb-4b6e5c8c6c24 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:57.570143 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ca0be-7114-4616-bfeb-4b6e5c8c6c24 None admin] PATH_INFO: `/v3/projects/23df208b3ac7427c970d4cda0a860820/users/bd010554e9fe44d88abb33f109f7fbd0/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:57.572067 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ca0be-7114-4616-bfeb-4b6e5c8c6c24 None admin] RBAC: Authorizing `identity:create_grant(project_id=23df208b3ac7427c970d4cda0a860820, user_id=bd010554e9fe44d88abb33f109f7fbd0, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:36:57.573548 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ca0be-7114-4616-bfeb-4b6e5c8c6c24 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:36:57.587177 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 437/871] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:36:57 2022] PUT /identity/v3/projects/23df208b3ac7427c970d4cda0a860820/users/bd010554e9fe44d88abb33f109f7fbd0/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:36:59.758602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933fe3a-4bcd-4361-b7f9-3773d1d741b7 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:59.759175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933fe3a-4bcd-4361-b7f9-3773d1d741b7 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:59.759175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933fe3a-4bcd-4361-b7f9-3773d1d741b7 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:59.760498 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 435/872] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:36:59.766060 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b9641-cf3d-4356-baa4-4310d9d5e200 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:36:59.766597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b9641-cf3d-4356-baa4-4310d9d5e200 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:36:59.766597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b9641-cf3d-4356-baa4-4310d9d5e200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:36:59.785881 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-a89b9641-cf3d-4356-baa4-4310d9d5e200 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:36:59.787728 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a89b9641-cf3d-4356-baa4-4310d9d5e200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:36:59.861494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 438/873] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:36:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:37:00.132106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933fe3a-4bcd-4361-b7f9-3773d1d741b7 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:00.235391 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933fe3a-4bcd-4361-b7f9-3773d1d741b7 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:00.253221 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:00.275115 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed239b63-e63a-4660-bfed-ce6f1e2e8037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:00.275749 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed239b63-e63a-4660-bfed-ce6f1e2e8037 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:00.276023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed239b63-e63a-4660-bfed-ce6f1e2e8037 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:00.276316 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed239b63-e63a-4660-bfed-ce6f1e2e8037 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:00.287119 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed239b63-e63a-4660-bfed-ce6f1e2e8037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:37:00.291821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-ed239b63-e63a-4660-bfed-ce6f1e2e8037 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:37:00.291821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:37:00.291821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:37:00.291821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:37:00.291821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:37:00.291821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:37:00.291821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:37:00.291821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:37:00.291821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:37:00.291821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:37:00.291821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:37:00.291821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:37:00.291821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:37:00.291821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:37:00.291821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:37:00.291821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:37:00.292801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:37:00.292801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:37:00.292801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:37:00.292801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:37:00.292801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:37:00.292801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:37:00.292801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:37:00.292801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:37:00.292801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:37:00.292801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:37:00.292801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:37:00.292801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:37:00.292801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:37:00.292801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:37:00.292801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:37:00.292801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:37:00.293793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:37:00.293793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:37:00.293793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:37:00.293793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:37:00.293793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:37:00.293793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:37:00.293793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:37:00.293793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:37:00.293793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:37:00.293793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:37:00.293793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:37:00.293793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:37:00.293793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 436/874] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:37:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:37:00.300369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b9641-cf3d-4356-baa4-4310d9d5e200 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:00.347893 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:00.407482 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b9641-cf3d-4356-baa4-4310d9d5e200 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:00.442752 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a844c-1abd-4e93-8615-235851b07951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:00.443349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a844c-1abd-4e93-8615-235851b07951 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:00.443623 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a844c-1abd-4e93-8615-235851b07951 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:00.443891 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a844c-1abd-4e93-8615-235851b07951 None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:00.444577 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a844c-1abd-4e93-8615-235851b07951 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:37:00.444875 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a844c-1abd-4e93-8615-235851b07951 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:00.446246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a844c-1abd-4e93-8615-235851b07951 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:00.454768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 439/875] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:37:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:37:00.460948 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed239b63-e63a-4660-bfed-ce6f1e2e8037 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:00.567699 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed239b63-e63a-4660-bfed-ce6f1e2e8037 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:00.603157 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793b0b1-c2a1-4f66-937f-05c9c9bf1775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:00.603793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793b0b1-c2a1-4f66-937f-05c9c9bf1775 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:00.604073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793b0b1-c2a1-4f66-937f-05c9c9bf1775 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:00.604372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793b0b1-c2a1-4f66-937f-05c9c9bf1775 None admin] PATH_INFO: `/v3/users/bd010554e9fe44d88abb33f109f7fbd0` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:00.605697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793b0b1-c2a1-4f66-937f-05c9c9bf1775 None admin] RBAC: Authorizing `identity:get_user(user_id=bd010554e9fe44d88abb33f109f7fbd0)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:00.607210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793b0b1-c2a1-4f66-937f-05c9c9bf1775 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:00.608855 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 437/876] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:37:00 2022] GET /identity/v3/users/bd010554e9fe44d88abb33f109f7fbd0 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:37:00.614843 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a844c-1abd-4e93-8615-235851b07951 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:00.718148 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a844c-1abd-4e93-8615-235851b07951 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:00.725467 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:00.754292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa42f57-b98d-4fe6-9a91-063f2f51739e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:00.754903 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa42f57-b98d-4fe6-9a91-063f2f51739e None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:00.755242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa42f57-b98d-4fe6-9a91-063f2f51739e None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:00.755518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa42f57-b98d-4fe6-9a91-063f2f51739e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:00.826539 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa42f57-b98d-4fe6-9a91-063f2f51739e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:00.828013 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa42f57-b98d-4fe6-9a91-063f2f51739e None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:00.929727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 440/877] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:37:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 30 00:37:00.936674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793b0b1-c2a1-4f66-937f-05c9c9bf1775 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:00.986120 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:01.042450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793b0b1-c2a1-4f66-937f-05c9c9bf1775 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:01.078880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e70015-d964-4be6-b390-14399bc71737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:01.079243 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e70015-d964-4be6-b390-14399bc71737 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:01.079523 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e70015-d964-4be6-b390-14399bc71737 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:01.079795 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e70015-d964-4be6-b390-14399bc71737 None admin] PATH_INFO: `/v3/projects/23df208b3ac7427c970d4cda0a860820` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:01.080966 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e70015-d964-4be6-b390-14399bc71737 None admin] RBAC: Authorizing `identity:get_project(project_id=23df208b3ac7427c970d4cda0a860820)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:01.082474 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e70015-d964-4be6-b390-14399bc71737 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:01.084145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 438/878] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:37:00 2022] GET /identity/v3/projects/23df208b3ac7427c970d4cda0a860820 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:37:01.090369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa42f57-b98d-4fe6-9a91-063f2f51739e None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:01.098733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:01.196400 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa42f57-b98d-4fe6-9a91-063f2f51739e None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:01.233623 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9968825-0acb-4cdc-88fd-eb4376067c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:01.234230 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9968825-0acb-4cdc-88fd-eb4376067c1d None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:01.234511 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9968825-0acb-4cdc-88fd-eb4376067c1d None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:01.234782 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9968825-0acb-4cdc-88fd-eb4376067c1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:01.235507 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9968825-0acb-4cdc-88fd-eb4376067c1d None admin] RBAC: Adding query filter params (role.id=980241d45fdb435d9f32c434057c71e4, scope.project.id=23df208b3ac7427c970d4cda0a860820, user.id=bd010554e9fe44d88abb33f109f7fbd0) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:37:01.235816 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9968825-0acb-4cdc-88fd-eb4376067c1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:01.237354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9968825-0acb-4cdc-88fd-eb4376067c1d None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:01.243388 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 441/879] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:37:01 2022] GET /identity/v3/role_assignments?user.id=bd010554e9fe44d88abb33f109f7fbd0&scope.project.id=23df208b3ac7427c970d4cda0a860820&role.id=980241d45fdb435d9f32c434057c71e4 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:37:03.453447 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a73eac-cd14-4ad4-9e21-aac3bf43870e None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:03.454007 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a73eac-cd14-4ad4-9e21-aac3bf43870e None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:03.454007 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a73eac-cd14-4ad4-9e21-aac3bf43870e None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:03.455339 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 439/880] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:37:03.460871 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0d689-f03e-4f04-8228-bae0484dac30 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:03.461336 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0d689-f03e-4f04-8228-bae0484dac30 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:03.461427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0d689-f03e-4f04-8228-bae0484dac30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:03.480416 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-56e0d689-f03e-4f04-8228-bae0484dac30 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:37:03.482332 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-56e0d689-f03e-4f04-8228-bae0484dac30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:37:03.546797 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:03.552290 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 442/881] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:37:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:37:03.825146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a73eac-cd14-4ad4-9e21-aac3bf43870e None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:03.929827 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a73eac-cd14-4ad4-9e21-aac3bf43870e None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:03.944229 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:03.964731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374f0df-3d9d-43a8-b994-309486f44e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:03.965341 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374f0df-3d9d-43a8-b994-309486f44e3d None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:03.965620 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374f0df-3d9d-43a8-b994-309486f44e3d None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:03.966572 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374f0df-3d9d-43a8-b994-309486f44e3d None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:03.967259 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374f0df-3d9d-43a8-b994-309486f44e3d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:03.968666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374f0df-3d9d-43a8-b994-309486f44e3d None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:03.988949 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 440/882] 10.222.0.36 () {60 vars in 1138 bytes} [Sun Oct 30 00:37:03 2022] POST /identity/v3/domains => generated 267 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:37:06.209864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a9e19-d4b3-4989-9228-2de99bad706a None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:06.210425 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a9e19-d4b3-4989-9228-2de99bad706a None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:06.210425 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a9e19-d4b3-4989-9228-2de99bad706a None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:06.211747 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 443/883] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:37:06.217744 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f685701-a837-4439-a474-0013053d16c7 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:06.218190 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f685701-a837-4439-a474-0013053d16c7 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:06.218303 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f685701-a837-4439-a474-0013053d16c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:06.238597 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-4f685701-a837-4439-a474-0013053d16c7 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:37:06.239797 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4f685701-a837-4439-a474-0013053d16c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:37:06.309234 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 441/884] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:37:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:37:06.583025 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a9e19-d4b3-4989-9228-2de99bad706a None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:06.694788 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a9e19-d4b3-4989-9228-2de99bad706a None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:06.730239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d22f7-0f4c-4887-b074-7db5368326dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:06.730876 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d22f7-0f4c-4887-b074-7db5368326dc None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:06.731218 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d22f7-0f4c-4887-b074-7db5368326dc None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:06.731490 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d22f7-0f4c-4887-b074-7db5368326dc None admin] PATH_INFO: `/v3/domains/d239eabe17194a73a8fc2f9e54f76550` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:06.742486 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d22f7-0f4c-4887-b074-7db5368326dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=d239eabe17194a73a8fc2f9e54f76550)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:06.743957 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d22f7-0f4c-4887-b074-7db5368326dc None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:06.745749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 444/885] 10.222.0.36 () {56 vars in 1183 bytes} [Sun Oct 30 00:37:06 2022] GET /identity/v3/domains/d239eabe17194a73a8fc2f9e54f76550 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:37:06.752855 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f685701-a837-4439-a474-0013053d16c7 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:06.853264 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f685701-a837-4439-a474-0013053d16c7 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:06.855486 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:06.889850 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2452b-b019-4568-b682-87a26fc95868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:06.890464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2452b-b019-4568-b682-87a26fc95868 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:06.890742 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2452b-b019-4568-b682-87a26fc95868 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:06.891013 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2452b-b019-4568-b682-87a26fc95868 None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:06.891680 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b2452b-b019-4568-b682-87a26fc95868 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:06.893148 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b2452b-b019-4568-b682-87a26fc95868 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:06.921508 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 442/886] 10.222.0.36 () {60 vars in 1141 bytes} [Sun Oct 30 00:37:06 2022] POST /identity/v3/projects => generated 361 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:37:09.128994 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a318011-5825-4f53-bc6f-9a4cbf669ee7 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:09.129512 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a318011-5825-4f53-bc6f-9a4cbf669ee7 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:09.129512 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a318011-5825-4f53-bc6f-9a4cbf669ee7 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:09.130815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 445/887] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:37:09.136226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af954bd8-9752-44af-8074-43a87f7cad46 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:09.136674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af954bd8-9752-44af-8074-43a87f7cad46 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:09.136777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af954bd8-9752-44af-8074-43a87f7cad46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:09.157705 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-af954bd8-9752-44af-8074-43a87f7cad46 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:37:09.158905 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-af954bd8-9752-44af-8074-43a87f7cad46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:37:09.226781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 443/888] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:37:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:37:09.498871 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a318011-5825-4f53-bc6f-9a4cbf669ee7 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:09.561530 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:09.605237 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a318011-5825-4f53-bc6f-9a4cbf669ee7 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:09.640956 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c8f35-9694-4826-93f2-74c0a2ea98ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:09.641584 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c8f35-9694-4826-93f2-74c0a2ea98ec None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:09.641864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c8f35-9694-4826-93f2-74c0a2ea98ec None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:09.642134 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c8f35-9694-4826-93f2-74c0a2ea98ec None admin] PATH_INFO: `/v3/domains/d239eabe17194a73a8fc2f9e54f76550` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:09.643260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c8f35-9694-4826-93f2-74c0a2ea98ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=d239eabe17194a73a8fc2f9e54f76550)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:09.644737 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c8f35-9694-4826-93f2-74c0a2ea98ec None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:09.646311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 446/889] 10.222.0.36 () {56 vars in 1183 bytes} [Sun Oct 30 00:37:09 2022] GET /identity/v3/domains/d239eabe17194a73a8fc2f9e54f76550 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:37:09.652302 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af954bd8-9752-44af-8074-43a87f7cad46 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:09.738212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:09.754339 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af954bd8-9752-44af-8074-43a87f7cad46 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:09.788368 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d357859-8438-4393-8d63-94081dcb76e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:09.788980 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d357859-8438-4393-8d63-94081dcb76e3 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:09.789257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d357859-8438-4393-8d63-94081dcb76e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:09.789530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d357859-8438-4393-8d63-94081dcb76e3 None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:09.790179 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d357859-8438-4393-8d63-94081dcb76e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:09.791618 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d357859-8438-4393-8d63-94081dcb76e3 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:09.814458 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 444/890] 10.222.0.36 () {60 vars in 1132 bytes} [Sun Oct 30 00:37:09 2022] POST /identity/v3/users => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:37:11.999122 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5144b9d-3db1-42d9-bfb2-092c0b2e00ac None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:11.999679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5144b9d-3db1-42d9-bfb2-092c0b2e00ac None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:11.999679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5144b9d-3db1-42d9-bfb2-092c0b2e00ac None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:12.001044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 447/891] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:37:12.006144 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7eaac-f18e-4fe1-96bc-a73b67927d81 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:12.006560 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7eaac-f18e-4fe1-96bc-a73b67927d81 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:12.006668 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7eaac-f18e-4fe1-96bc-a73b67927d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:12.024640 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-2aa7eaac-f18e-4fe1-96bc-a73b67927d81 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:37:12.025830 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-2aa7eaac-f18e-4fe1-96bc-a73b67927d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:37:12.091762 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 445/892] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:37:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:37:12.363095 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5144b9d-3db1-42d9-bfb2-092c0b2e00ac None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:12.462224 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5144b9d-3db1-42d9-bfb2-092c0b2e00ac None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:12.488369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:12.496196 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9e6a0-2831-460e-aad7-1f01f5321cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:12.496996 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9e6a0-2831-460e-aad7-1f01f5321cb0 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:12.497149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9e6a0-2831-460e-aad7-1f01f5321cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:12.497423 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9e6a0-2831-460e-aad7-1f01f5321cb0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:12.507259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca9e6a0-2831-460e-aad7-1f01f5321cb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:37:12.512505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-0ca9e6a0-2831-460e-aad7-1f01f5321cb0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:37:12.512505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:37:12.512505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:37:12.512505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:37:12.512505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:37:12.512505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:37:12.512505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:37:12.512505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:37:12.512505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:37:12.512505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:37:12.512505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:37:12.512505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:37:12.512505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:37:12.512505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:37:12.512505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:37:12.512505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:37:12.514035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:37:12.514035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:37:12.514035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:37:12.514035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:37:12.514035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:37:12.514035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:37:12.514035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:37:12.514035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:37:12.514035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:37:12.514035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:37:12.514035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:37:12.514035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:37:12.514035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:37:12.514035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:37:12.514035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:37:12.514035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:37:12.514902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:37:12.514902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:37:12.514902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:37:12.514902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:37:12.514902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:37:12.514902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:37:12.514902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:37:12.514902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:37:12.514902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:37:12.514902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:37:12.514902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:37:12.514902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:37:12.514902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 448/893] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:37:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:37:12.519844 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7eaac-f18e-4fe1-96bc-a73b67927d81 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:12.586828 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:12.618273 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7eaac-f18e-4fe1-96bc-a73b67927d81 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:12.651550 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f75c8f-3910-4f62-89e5-1d041464a8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:12.652220 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f75c8f-3910-4f62-89e5-1d041464a8b9 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:12.652484 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f75c8f-3910-4f62-89e5-1d041464a8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:12.653374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f75c8f-3910-4f62-89e5-1d041464a8b9 None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:12.653960 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f75c8f-3910-4f62-89e5-1d041464a8b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:37:12.654267 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f75c8f-3910-4f62-89e5-1d041464a8b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:12.655636 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f75c8f-3910-4f62-89e5-1d041464a8b9 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:12.664901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 446/894] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:37:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:37:12.671579 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9e6a0-2831-460e-aad7-1f01f5321cb0 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:12.770388 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9e6a0-2831-460e-aad7-1f01f5321cb0 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:12.804304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c57af8-cd42-4cba-8460-fbf369b7fcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:12.804946 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c57af8-cd42-4cba-8460-fbf369b7fcfc None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:12.805211 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c57af8-cd42-4cba-8460-fbf369b7fcfc None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:12.805486 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c57af8-cd42-4cba-8460-fbf369b7fcfc None admin] PATH_INFO: `/v3/users/e5b767df163e449ab4707075e636658c` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:12.819371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c57af8-cd42-4cba-8460-fbf369b7fcfc None admin] RBAC: Authorizing `identity:get_user(user_id=e5b767df163e449ab4707075e636658c)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:12.820853 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c57af8-cd42-4cba-8460-fbf369b7fcfc None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:12.822447 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 449/895] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:37:12 2022] GET /identity/v3/users/e5b767df163e449ab4707075e636658c => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:37:12.828722 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f75c8f-3910-4f62-89e5-1d041464a8b9 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:12.927860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f75c8f-3910-4f62-89e5-1d041464a8b9 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:12.960787 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ceda2-25d9-45cf-a446-8c3131162857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:12.961414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ceda2-25d9-45cf-a446-8c3131162857 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:12.961692 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ceda2-25d9-45cf-a446-8c3131162857 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:12.961965 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ceda2-25d9-45cf-a446-8c3131162857 None admin] PATH_INFO: `/v3/projects/dd5bd1cec2e44ee4833915298be0a6e0` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:12.963071 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ceda2-25d9-45cf-a446-8c3131162857 None admin] RBAC: Authorizing `identity:get_project(project_id=dd5bd1cec2e44ee4833915298be0a6e0)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:12.964537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ceda2-25d9-45cf-a446-8c3131162857 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:12.966727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 447/896] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:37:12 2022] GET /identity/v3/projects/dd5bd1cec2e44ee4833915298be0a6e0 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:37:12.972776 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c57af8-cd42-4cba-8460-fbf369b7fcfc None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:13.017990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:13.069823 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c57af8-cd42-4cba-8460-fbf369b7fcfc None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:13.102938 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1b433-9450-45ee-ba35-0a60e1d68b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:13.103584 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1b433-9450-45ee-ba35-0a60e1d68b66 None admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:13.103866 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1b433-9450-45ee-ba35-0a60e1d68b66 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:13.104163 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1b433-9450-45ee-ba35-0a60e1d68b66 None admin] PATH_INFO: `/v3/projects/dd5bd1cec2e44ee4833915298be0a6e0/users/e5b767df163e449ab4707075e636658c/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:13.106161 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1b433-9450-45ee-ba35-0a60e1d68b66 None admin] RBAC: Authorizing `identity:create_grant(project_id=dd5bd1cec2e44ee4833915298be0a6e0, user_id=e5b767df163e449ab4707075e636658c, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:13.107588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1b433-9450-45ee-ba35-0a60e1d68b66 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:13.120835 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 450/897] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:37:12 2022] PUT /identity/v3/projects/dd5bd1cec2e44ee4833915298be0a6e0/users/e5b767df163e449ab4707075e636658c/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:37:15.308594 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237e2a7-9734-469e-9222-c21072c5be95 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:15.309119 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237e2a7-9734-469e-9222-c21072c5be95 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:15.309723 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237e2a7-9734-469e-9222-c21072c5be95 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:15.311061 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 448/898] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:37:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:37:15.316338 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff4e0e-dcc3-492f-aef6-c2e4138bdc32 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:15.316706 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff4e0e-dcc3-492f-aef6-c2e4138bdc32 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:15.316885 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff4e0e-dcc3-492f-aef6-c2e4138bdc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:15.335103 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-1aff4e0e-dcc3-492f-aef6-c2e4138bdc32 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:37:15.336315 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-1aff4e0e-dcc3-492f-aef6-c2e4138bdc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:37:15.408662 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 451/899] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:37:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:37:15.679870 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237e2a7-9734-469e-9222-c21072c5be95 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:15.682861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:15.786374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237e2a7-9734-469e-9222-c21072c5be95 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:15.822467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06de9b-6c7f-44c5-b53f-6ff9f847f842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:15.823093 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06de9b-6c7f-44c5-b53f-6ff9f847f842 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:15.823371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06de9b-6c7f-44c5-b53f-6ff9f847f842 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:15.823640 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06de9b-6c7f-44c5-b53f-6ff9f847f842 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:15.834419 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06de9b-6c7f-44c5-b53f-6ff9f847f842 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:37:15.839222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-ae06de9b-6c7f-44c5-b53f-6ff9f847f842 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:37:15.839222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:37:15.839222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:37:15.839222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:37:15.839222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:37:15.839222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:37:15.839222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:37:15.839222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:37:15.839222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:37:15.839222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:37:15.839222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:37:15.839222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:37:15.839222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:37:15.839222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:37:15.839222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:37:15.839222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:37:15.840541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:37:15.840541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:37:15.840541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:37:15.840541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:37:15.840541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:37:15.840541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:37:15.840541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:37:15.840541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:37:15.840541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:37:15.840541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:37:15.840541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:37:15.840541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:37:15.840541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:37:15.840541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:37:15.840541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:37:15.840541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:37:15.841539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:37:15.841539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:37:15.841539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:37:15.841539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:37:15.841539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:37:15.841539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:37:15.841539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:37:15.841539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:37:15.841539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:37:15.841539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:37:15.841539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:37:15.841539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:37:15.841539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 449/900] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:37:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:37:15.846666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff4e0e-dcc3-492f-aef6-c2e4138bdc32 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:15.881498 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:15.948465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff4e0e-dcc3-492f-aef6-c2e4138bdc32 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:15.985544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54563c-e956-48a0-b79b-bbde7e54e40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:15.986152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54563c-e956-48a0-b79b-bbde7e54e40d None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:15.986427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54563c-e956-48a0-b79b-bbde7e54e40d None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:15.986695 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54563c-e956-48a0-b79b-bbde7e54e40d None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:15.987351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54563c-e956-48a0-b79b-bbde7e54e40d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:37:15.987649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54563c-e956-48a0-b79b-bbde7e54e40d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:15.989049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54563c-e956-48a0-b79b-bbde7e54e40d None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:15.998228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 452/901] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:37:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:37:16.004247 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06de9b-6c7f-44c5-b53f-6ff9f847f842 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:16.066201 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:16.105020 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06de9b-6c7f-44c5-b53f-6ff9f847f842 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:16.139162 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaaa77-2b3e-4ea5-b9be-c1323f9270b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:16.139788 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaaa77-2b3e-4ea5-b9be-c1323f9270b6 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:16.140063 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaaa77-2b3e-4ea5-b9be-c1323f9270b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:16.140365 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaaa77-2b3e-4ea5-b9be-c1323f9270b6 None admin] PATH_INFO: `/v3/users/e5b767df163e449ab4707075e636658c` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:16.141475 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aaaa77-2b3e-4ea5-b9be-c1323f9270b6 None admin] RBAC: Authorizing `identity:get_user(user_id=e5b767df163e449ab4707075e636658c)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:16.142900 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aaaa77-2b3e-4ea5-b9be-c1323f9270b6 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:16.144512 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 450/902] 10.222.0.36 () {56 vars in 1177 bytes} [Sun Oct 30 00:37:16 2022] GET /identity/v3/users/e5b767df163e449ab4707075e636658c => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:37:16.150373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54563c-e956-48a0-b79b-bbde7e54e40d None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:16.256675 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:16.260696 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54563c-e956-48a0-b79b-bbde7e54e40d None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:16.296224 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc4319-a781-4002-a0d7-02d223526491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:16.296830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc4319-a781-4002-a0d7-02d223526491 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:16.297111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc4319-a781-4002-a0d7-02d223526491 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:16.297376 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc4319-a781-4002-a0d7-02d223526491 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:16.368835 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc4319-a781-4002-a0d7-02d223526491 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:16.370309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc4319-a781-4002-a0d7-02d223526491 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:16.469161 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:16.474288 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 453/903] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:37:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 30 00:37:16.479885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaaa77-2b3e-4ea5-b9be-c1323f9270b6 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:16.581509 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaaa77-2b3e-4ea5-b9be-c1323f9270b6 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:16.609570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:16.615663 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f620a81-2af5-44ea-a541-c0b08562b304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:16.616326 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f620a81-2af5-44ea-a541-c0b08562b304 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:16.616607 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f620a81-2af5-44ea-a541-c0b08562b304 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:16.616879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f620a81-2af5-44ea-a541-c0b08562b304 None admin] PATH_INFO: `/v3/projects/dd5bd1cec2e44ee4833915298be0a6e0` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:16.617994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f620a81-2af5-44ea-a541-c0b08562b304 None admin] RBAC: Authorizing `identity:get_project(project_id=dd5bd1cec2e44ee4833915298be0a6e0)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:16.619430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f620a81-2af5-44ea-a541-c0b08562b304 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:16.621590 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 451/904] 10.222.0.36 () {56 vars in 1186 bytes} [Sun Oct 30 00:37:16 2022] GET /identity/v3/projects/dd5bd1cec2e44ee4833915298be0a6e0 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:37:16.628808 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc4319-a781-4002-a0d7-02d223526491 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:37:16.733522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc4319-a781-4002-a0d7-02d223526491 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:37:16.770895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35437eea-634a-4752-8339-8a743df01ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:37:16.771501 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35437eea-634a-4752-8339-8a743df01ad6 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:37:16.771776 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35437eea-634a-4752-8339-8a743df01ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:37:16.772044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35437eea-634a-4752-8339-8a743df01ad6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:37:16.772795 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35437eea-634a-4752-8339-8a743df01ad6 None admin] RBAC: Adding query filter params (role.id=980241d45fdb435d9f32c434057c71e4, scope.project.id=dd5bd1cec2e44ee4833915298be0a6e0, user.id=e5b767df163e449ab4707075e636658c) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:37:16.773099 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35437eea-634a-4752-8339-8a743df01ad6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:37:16.774536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35437eea-634a-4752-8339-8a743df01ad6 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:37:16.786500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 454/905] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:37:16 2022] GET /identity/v3/role_assignments?user.id=e5b767df163e449ab4707075e636658c&scope.project.id=dd5bd1cec2e44ee4833915298be0a6e0&role.id=980241d45fdb435d9f32c434057c71e4 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:37:26.614717 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:37:26.777886 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=69 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:14.021551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46158873-679e-4578-82df-7966b9f308ee None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:14.022502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46158873-679e-4578-82df-7966b9f308ee None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:14.022502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46158873-679e-4578-82df-7966b9f308ee None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:14.023604 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 452/906] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:38:14.032501 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4d734-b469-4a3b-aad8-2908a4df712c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:14.033008 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4d734-b469-4a3b-aad8-2908a4df712c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:14.033094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4d734-b469-4a3b-aad8-2908a4df712c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:14.055639 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-77e4d734-b469-4a3b-aad8-2908a4df712c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:38:14.058380 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-77e4d734-b469-4a3b-aad8-2908a4df712c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:38:14.157350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 455/907] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:38:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:38:14.457259 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46158873-679e-4578-82df-7966b9f308ee None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:14.613632 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46158873-679e-4578-82df-7966b9f308ee None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:14.675057 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb25a85-d57a-4de3-8f01-803437f351b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:14.675803 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb25a85-d57a-4de3-8f01-803437f351b1 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:14.676126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb25a85-d57a-4de3-8f01-803437f351b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:14.676407 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb25a85-d57a-4de3-8f01-803437f351b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:14.689194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb25a85-d57a-4de3-8f01-803437f351b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:14.690805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb25a85-d57a-4de3-8f01-803437f351b1 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:14.715779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-bbb25a85-d57a-4de3-8f01-803437f351b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:38:14.715779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:14.715779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:14.715779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:14.715779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:14.715779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:14.715779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:14.715779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:14.715779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:14.715779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:14.715779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:14.715779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:14.715779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:38:14.715779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:38:14.715779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:38:14.715779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:38:14.717123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:14.717123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:14.717123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:14.717123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:14.717123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:14.717123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:14.717123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:14.717123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:14.717123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:14.717123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:14.717123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:14.717123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:14.717123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:14.717123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:14.717123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:14.717123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:14.717123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:38:14.718126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:38:14.718126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:38:14.718126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:38:14.718126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 453/908] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:14.723447 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4d734-b469-4a3b-aad8-2908a4df712c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:14.870987 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4d734-b469-4a3b-aad8-2908a4df712c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:14.913335 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:14.924866 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4eaab-692f-4f57-b166-ee0101161fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:14.925669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4eaab-692f-4f57-b166-ee0101161fd1 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:14.926017 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4eaab-692f-4f57-b166-ee0101161fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:14.926302 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4eaab-692f-4f57-b166-ee0101161fd1 None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:14.926952 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4eaab-692f-4f57-b166-ee0101161fd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:14.927249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4eaab-692f-4f57-b166-ee0101161fd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:14.928882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4eaab-692f-4f57-b166-ee0101161fd1 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:14.945049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 456/909] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:14.951724 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb25a85-d57a-4de3-8f01-803437f351b1 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:15.036023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:15.104955 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb25a85-d57a-4de3-8f01-803437f351b1 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:15.158313 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24faf691-293b-4a74-bb8f-469470c337d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:15.159121 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24faf691-293b-4a74-bb8f-469470c337d8 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:15.159395 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24faf691-293b-4a74-bb8f-469470c337d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:15.159670 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24faf691-293b-4a74-bb8f-469470c337d8 None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:15.160383 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24faf691-293b-4a74-bb8f-469470c337d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:15.161831 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24faf691-293b-4a74-bb8f-469470c337d8 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:15.187181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 454/910] 10.222.0.36 () {60 vars in 1132 bytes} [Sun Oct 30 00:38:14 2022] POST /identity/v3/users => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:38:17.506532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9510034-cbf7-42d2-b407-781ac5ed0ab5 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:17.507134 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9510034-cbf7-42d2-b407-781ac5ed0ab5 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:17.507134 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9510034-cbf7-42d2-b407-781ac5ed0ab5 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:17.508641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 457/911] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:38:17.517960 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb16866-8abf-464c-b01d-3d78b7cf044e None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:17.517960 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb16866-8abf-464c-b01d-3d78b7cf044e None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:17.517960 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb16866-8abf-464c-b01d-3d78b7cf044e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:17.535391 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-ccb16866-8abf-464c-b01d-3d78b7cf044e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:38:17.536683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ccb16866-8abf-464c-b01d-3d78b7cf044e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:38:17.624139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 455/912] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:38:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:38:17.897640 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9510034-cbf7-42d2-b407-781ac5ed0ab5 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:18.015650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9510034-cbf7-42d2-b407-781ac5ed0ab5 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:18.068015 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb566e2-9d89-4997-991f-258979187b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:18.068690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb566e2-9d89-4997-991f-258979187b7d None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:18.068994 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb566e2-9d89-4997-991f-258979187b7d None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:18.069343 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb566e2-9d89-4997-991f-258979187b7d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:18.086258 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb566e2-9d89-4997-991f-258979187b7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:38:18.093286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-0fb566e2-9d89-4997-991f-258979187b7d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:38:18.093286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:18.093286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:18.093286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:18.093286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:18.093286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:18.093286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:18.093286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:18.093286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:18.093286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:18.093286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:18.093286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:18.093286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:38:18.093286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:38:18.093286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:38:18.093286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:38:18.094306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:38:18.094306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:38:18.094306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:38:18.094306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:38:18.094306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:18.094306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:18.094306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:18.094306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:18.094306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:18.094306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:18.094306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:18.094306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:18.094306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:18.094306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:18.094306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:18.094306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:18.095852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:18.095852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:18.095852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:18.095852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:18.095852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:38:18.095852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:38:18.095852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:38:18.095852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:38:18.095852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:38:18.095852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:38:18.095852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:38:18.095852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:18.095852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 458/913] 10.222.0.36 () {56 vars in 1102 bytes} [Sun Oct 30 00:38:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:18.101099 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb16866-8abf-464c-b01d-3d78b7cf044e None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:18.147860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:18.220340 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb16866-8abf-464c-b01d-3d78b7cf044e None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:18.261269 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40633b00-ef8a-4ae9-be3d-321f35f49bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:18.261886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40633b00-ef8a-4ae9-be3d-321f35f49bad None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:18.262168 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40633b00-ef8a-4ae9-be3d-321f35f49bad None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:18.262448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40633b00-ef8a-4ae9-be3d-321f35f49bad None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:18.263108 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40633b00-ef8a-4ae9-be3d-321f35f49bad None admin] RBAC: Adding query filter params (name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:18.263400 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40633b00-ef8a-4ae9-be3d-321f35f49bad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:18.265122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40633b00-ef8a-4ae9-be3d-321f35f49bad None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:18.273926 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 456/914] 10.222.0.36 () {56 vars in 1103 bytes} [Sun Oct 30 00:38:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:18.281355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb566e2-9d89-4997-991f-258979187b7d None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:18.340641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:18.406896 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb566e2-9d89-4997-991f-258979187b7d None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:18.449656 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f3b41-d7e9-4d00-9c3e-b7dbd6e7bd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:18.450309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f3b41-d7e9-4d00-9c3e-b7dbd6e7bd9b None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:18.450572 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f3b41-d7e9-4d00-9c3e-b7dbd6e7bd9b None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:18.450840 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f3b41-d7e9-4d00-9c3e-b7dbd6e7bd9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:18.462216 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f3b41-d7e9-4d00-9c3e-b7dbd6e7bd9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:18.464845 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f3b41-d7e9-4d00-9c3e-b7dbd6e7bd9b None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:18.481950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-382f3b41-d7e9-4d00-9c3e-b7dbd6e7bd9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:38:18.481950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:18.481950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:18.481950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:18.481950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:18.481950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:18.481950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:18.481950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:18.481950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:18.481950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:18.481950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:18.481950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:18.481950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:38:18.481950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:38:18.481950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:38:18.481950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:38:18.483054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:18.483054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:18.483054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:18.483054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:18.483054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:18.483054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:18.483054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:18.483054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:18.483054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:18.483054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:18.483054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:18.483054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:18.483054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:18.483054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:18.483054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:18.483054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:18.483054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:38:18.484416 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:38:18.484416 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:38:18.484416 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:18.484416 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 459/915] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:18.489167 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40633b00-ef8a-4ae9-be3d-321f35f49bad None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:18.601133 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:18.604712 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40633b00-ef8a-4ae9-be3d-321f35f49bad None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:18.645799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3fcd8-b340-4906-80ad-92e2ae0b8dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:18.646505 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3fcd8-b340-4906-80ad-92e2ae0b8dcd None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:18.646855 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3fcd8-b340-4906-80ad-92e2ae0b8dcd None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:18.647130 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3fcd8-b340-4906-80ad-92e2ae0b8dcd None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:18.647905 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3fcd8-b340-4906-80ad-92e2ae0b8dcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:18.648299 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3fcd8-b340-4906-80ad-92e2ae0b8dcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:18.650077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3fcd8-b340-4906-80ad-92e2ae0b8dcd None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:18.665272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 457/916] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:18.672053 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f3b41-d7e9-4d00-9c3e-b7dbd6e7bd9b None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:18.790654 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f3b41-d7e9-4d00-9c3e-b7dbd6e7bd9b None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:18.792823 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:18.835031 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a25c41-bfc1-4429-bf13-34f3c636f2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:18.836711 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a25c41-bfc1-4429-bf13-34f3c636f2f5 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:18.836711 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a25c41-bfc1-4429-bf13-34f3c636f2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:18.836711 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a25c41-bfc1-4429-bf13-34f3c636f2f5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:18.844811 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a25c41-bfc1-4429-bf13-34f3c636f2f5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:18.846390 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a25c41-bfc1-4429-bf13-34f3c636f2f5 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:18.858039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-42a25c41-bfc1-4429-bf13-34f3c636f2f5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 30 00:38:18.858039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:18.858039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:18.858039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:18.858039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:18.858039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:18.858039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:18.858039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:18.858039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:18.858039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:18.858039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:18.858039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:18.858039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:38:18.858039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:38:18.858039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:38:18.858039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:38:18.859057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:18.859057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:18.859057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:38:18.859057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:38:18.859057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:38:18.859057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:38:18.859057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:18.859057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:18.859057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:18.859057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:18.859057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:18.859057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:18.859057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:18.859057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:18.859057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:18.859057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:18.859057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:18.859057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:18.860044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:18.860044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:18.860044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:38:18.860044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:38:18.860044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:38:18.860044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:38:18.860044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:38:18.860044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:38:18.860044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 30 00:38:18.860044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:18.860044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 460/917] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:38:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:18.867182 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3fcd8-b340-4906-80ad-92e2ae0b8dcd None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:19.012799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3fcd8-b340-4906-80ad-92e2ae0b8dcd None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:19.053688 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5d06e-6d9b-4781-ac93-c60861704c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:19.054430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5d06e-6d9b-4781-ac93-c60861704c93 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:19.054711 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5d06e-6d9b-4781-ac93-c60861704c93 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:19.055023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5d06e-6d9b-4781-ac93-c60861704c93 None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:19.055800 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5d06e-6d9b-4781-ac93-c60861704c93 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:19.056127 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5d06e-6d9b-4781-ac93-c60861704c93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:19.057712 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5d06e-6d9b-4781-ac93-c60861704c93 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:19.078136 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 458/918] 10.222.0.36 () {56 vars in 1143 bytes} [Sun Oct 30 00:38:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:38:19.083798 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a25c41-bfc1-4429-bf13-34f3c636f2f5 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:19.203856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a25c41-bfc1-4429-bf13-34f3c636f2f5 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:19.247816 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5b957-0a5b-4ec1-8c21-7dbe03985016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:19.247816 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5b957-0a5b-4ec1-8c21-7dbe03985016 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:19.247816 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5b957-0a5b-4ec1-8c21-7dbe03985016 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:19.247816 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e5b957-0a5b-4ec1-8c21-7dbe03985016 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:19.262628 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5b957-0a5b-4ec1-8c21-7dbe03985016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:19.264256 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e5b957-0a5b-4ec1-8c21-7dbe03985016 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:19.281585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-15e5b957-0a5b-4ec1-8c21-7dbe03985016 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:38:19.281585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:19.281585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:19.281585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:19.281585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:19.281585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:19.281585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:19.281585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:19.281585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:19.281585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:19.281585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:19.281585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:19.281585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:38:19.281585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:38:19.281585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:38:19.281585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:38:19.282674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:19.282674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:19.282674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:19.282674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:19.282674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:19.282674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:19.282674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:19.282674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:19.282674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:19.282674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:19.282674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:19.282674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:19.282674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:19.282674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:19.282674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:19.282674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:19.282674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:38:19.284164 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:38:19.284164 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:38:19.284164 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:19.284164 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 461/919] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:19.289285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5d06e-6d9b-4781-ac93-c60861704c93 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:19.311041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:19.435014 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5d06e-6d9b-4781-ac93-c60861704c93 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:19.475426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14f1c7-7481-4988-95f6-d97b0617bed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:19.476044 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14f1c7-7481-4988-95f6-d97b0617bed5 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:19.476354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14f1c7-7481-4988-95f6-d97b0617bed5 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:19.476627 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14f1c7-7481-4988-95f6-d97b0617bed5 None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:19.477270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14f1c7-7481-4988-95f6-d97b0617bed5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:19.477577 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14f1c7-7481-4988-95f6-d97b0617bed5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:19.479042 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14f1c7-7481-4988-95f6-d97b0617bed5 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:19.491391 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 459/920] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:19.497962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5b957-0a5b-4ec1-8c21-7dbe03985016 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:19.526478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:19.639508 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e5b957-0a5b-4ec1-8c21-7dbe03985016 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:19.692204 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4d379-a86f-46a8-b6ca-16a459d43de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:19.692917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4d379-a86f-46a8-b6ca-16a459d43de7 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:19.693117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4d379-a86f-46a8-b6ca-16a459d43de7 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:19.693391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4d379-a86f-46a8-b6ca-16a459d43de7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:19.702624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df4d379-a86f-46a8-b6ca-16a459d43de7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:19.704132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df4d379-a86f-46a8-b6ca-16a459d43de7 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:19.717562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-2df4d379-a86f-46a8-b6ca-16a459d43de7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:38:19.717562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:19.717562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:19.717562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:19.717562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:19.717562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:19.717562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:19.717562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:19.717562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:19.717562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:19.717562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:19.717562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:19.717562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:38:19.717562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:38:19.717562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:38:19.718940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:38:19.718940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:19.718940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:19.718940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:19.718940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:19.718940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:19.718940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:19.718940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:19.718940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:19.718940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:19.718940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:19.718940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:19.718940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:19.718940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:19.718940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:19.718940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:19.718940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:19.719982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:38:19.719982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:38:19.719982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:38:19.719982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:38:19.719982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:38:19.719982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:38:19.719982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:38:19.719982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:19.719982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 462/921] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:38:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:19.725480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14f1c7-7481-4988-95f6-d97b0617bed5 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:19.823366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:19.849242 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14f1c7-7481-4988-95f6-d97b0617bed5 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:19.898651 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f170687-7ad3-42fc-8be9-8d543746638a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:19.899328 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f170687-7ad3-42fc-8be9-8d543746638a None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:19.899611 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f170687-7ad3-42fc-8be9-8d543746638a None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:19.900008 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f170687-7ad3-42fc-8be9-8d543746638a None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:19.900799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f170687-7ad3-42fc-8be9-8d543746638a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:19.901156 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f170687-7ad3-42fc-8be9-8d543746638a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:19.902708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f170687-7ad3-42fc-8be9-8d543746638a None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:19.918146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 460/922] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:38:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:19.924982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4d379-a86f-46a8-b6ca-16a459d43de7 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:20.048751 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:20.094909 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4d379-a86f-46a8-b6ca-16a459d43de7 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:20.144620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbef76-74a9-4f1b-81af-cd8a48aa96c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:20.145570 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbef76-74a9-4f1b-81af-cd8a48aa96c9 None admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:20.146007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbef76-74a9-4f1b-81af-cd8a48aa96c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:20.146488 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbef76-74a9-4f1b-81af-cd8a48aa96c9 None admin] PATH_INFO: `/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/5ded89d48d6c4a15a5c5829f594f8776/roles/9a8de43206fa4a8fa4901488bf86f9f5` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:20.168724 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbef76-74a9-4f1b-81af-cd8a48aa96c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab3a0e5fc5a344739b58ec8313639496, user_id=5ded89d48d6c4a15a5c5829f594f8776, role_id=9a8de43206fa4a8fa4901488bf86f9f5)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:20.171454 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbef76-74a9-4f1b-81af-cd8a48aa96c9 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:20.193964 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 463/923] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:38:19 2022] PUT /identity/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/5ded89d48d6c4a15a5c5829f594f8776/roles/9a8de43206fa4a8fa4901488bf86f9f5 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:38:22.524456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11f467-ac8d-487c-8792-3f591e6ed53f None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:22.525147 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11f467-ac8d-487c-8792-3f591e6ed53f None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:22.525147 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11f467-ac8d-487c-8792-3f591e6ed53f None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:22.526719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 461/924] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:38:22.533005 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfdb57-b8bc-403f-8e6f-1e0c99fdd517 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:22.533578 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfdb57-b8bc-403f-8e6f-1e0c99fdd517 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:22.533578 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfdb57-b8bc-403f-8e6f-1e0c99fdd517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:22.555478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-6ccfdb57-b8bc-403f-8e6f-1e0c99fdd517 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:38:22.557381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-6ccfdb57-b8bc-403f-8e6f-1e0c99fdd517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:38:22.636187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 464/925] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:38:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 30 00:38:22.930223 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11f467-ac8d-487c-8792-3f591e6ed53f None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:23.049032 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11f467-ac8d-487c-8792-3f591e6ed53f None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:23.086544 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e60f5f-49a7-42d9-8dec-269dfb19e5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:23.087182 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e60f5f-49a7-42d9-8dec-269dfb19e5a1 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:23.087529 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e60f5f-49a7-42d9-8dec-269dfb19e5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:23.087806 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e60f5f-49a7-42d9-8dec-269dfb19e5a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:23.095145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:23.099448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e60f5f-49a7-42d9-8dec-269dfb19e5a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:38:23.107232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-67e60f5f-49a7-42d9-8dec-269dfb19e5a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:38:23.107232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:23.107232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:23.107232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:23.107232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:23.107232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:23.107232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:23.107232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:23.107232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:23.107232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:23.107232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:23.107232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:23.107232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:38:23.107232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:38:23.107232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:38:23.107232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:38:23.108579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:38:23.108579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:38:23.108579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:38:23.108579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:38:23.108579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:23.108579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:23.108579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:23.108579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:23.108579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:23.108579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:23.108579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:23.108579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:23.108579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:23.108579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:23.108579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:23.108579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:23.109502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:23.109502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:23.109502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:23.109502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:23.109502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:38:23.109502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:38:23.109502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:38:23.109502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:38:23.109502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:38:23.109502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:38:23.109502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 30 00:38:23.109502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:38:23.112535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 462/926] 10.222.0.36 () {56 vars in 1102 bytes} [Sun Oct 30 00:38:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:23.115151 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfdb57-b8bc-403f-8e6f-1e0c99fdd517 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:23.177328 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:23.228924 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfdb57-b8bc-403f-8e6f-1e0c99fdd517 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:23.265414 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd5914-90ad-46f0-8d34-de4c7955348e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:23.266052 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd5914-90ad-46f0-8d34-de4c7955348e None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:23.266333 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd5914-90ad-46f0-8d34-de4c7955348e None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:23.266751 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd5914-90ad-46f0-8d34-de4c7955348e None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:23.267391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd5914-90ad-46f0-8d34-de4c7955348e None admin] RBAC: Adding query filter params (name=service) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:23.267773 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd5914-90ad-46f0-8d34-de4c7955348e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:23.269696 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd5914-90ad-46f0-8d34-de4c7955348e None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:23.280733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 465/927] 10.222.0.36 () {56 vars in 1103 bytes} [Sun Oct 30 00:38:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:23.287039 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e60f5f-49a7-42d9-8dec-269dfb19e5a1 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:23.413581 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e60f5f-49a7-42d9-8dec-269dfb19e5a1 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:23.454523 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b102711-f464-4fb7-b9ed-2334c45dddfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:23.455101 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b102711-f464-4fb7-b9ed-2334c45dddfa None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:23.455453 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b102711-f464-4fb7-b9ed-2334c45dddfa None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:23.455732 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b102711-f464-4fb7-b9ed-2334c45dddfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:23.468719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b102711-f464-4fb7-b9ed-2334c45dddfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:23.470362 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b102711-f464-4fb7-b9ed-2334c45dddfa None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:23.486734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-5b102711-f464-4fb7-b9ed-2334c45dddfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:38:23.486734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:23.486734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:23.486734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:23.486734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:23.486734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:23.486734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:23.486734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:23.486734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:23.486734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:23.486734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:23.486734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:23.486734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:38:23.486734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:38:23.486734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:38:23.486734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:38:23.487933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:23.487933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:23.487933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:23.487933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:23.487933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:23.487933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:23.487933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:23.487933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:23.487933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:23.487933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:23.487933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:23.487933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:23.487933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:23.487933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:23.487933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:23.487933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:23.487933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:38:23.488882 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:38:23.488882 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:38:23.488882 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:38:23.488882 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 463/928] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:23.495339 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd5914-90ad-46f0-8d34-de4c7955348e None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:23.615735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd5914-90ad-46f0-8d34-de4c7955348e None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:23.636343 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:23.656659 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c69cb7-b162-4215-8cb8-3b1e5668a039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:23.657268 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c69cb7-b162-4215-8cb8-3b1e5668a039 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:23.657548 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c69cb7-b162-4215-8cb8-3b1e5668a039 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:23.657870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c69cb7-b162-4215-8cb8-3b1e5668a039 None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:23.658505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c69cb7-b162-4215-8cb8-3b1e5668a039 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:23.658887 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c69cb7-b162-4215-8cb8-3b1e5668a039 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:23.660250 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c69cb7-b162-4215-8cb8-3b1e5668a039 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:23.673207 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 466/929] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:23.679613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b102711-f464-4fb7-b9ed-2334c45dddfa None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:23.736522 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:23.792491 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b102711-f464-4fb7-b9ed-2334c45dddfa None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:23.830970 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402db913-4a82-4bac-a21d-22a4679c3a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:23.831653 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402db913-4a82-4bac-a21d-22a4679c3a40 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:23.831937 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402db913-4a82-4bac-a21d-22a4679c3a40 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:23.832270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402db913-4a82-4bac-a21d-22a4679c3a40 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:23.840811 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402db913-4a82-4bac-a21d-22a4679c3a40 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:23.842364 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402db913-4a82-4bac-a21d-22a4679c3a40 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:23.855602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-402db913-4a82-4bac-a21d-22a4679c3a40 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 30 00:38:23.855602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:23.855602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:23.855602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:23.855602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:23.855602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:23.855602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:23.855602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:23.855602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:23.855602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:23.855602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:23.855602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:23.855602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:38:23.855602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:38:23.855602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:38:23.855602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:38:23.857459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:23.857459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:23.857459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:38:23.857459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:38:23.857459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:38:23.857459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:38:23.857459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:23.857459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:23.857459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:23.857459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:23.857459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:23.857459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:23.857459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:23.857459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:23.857459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:23.857459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:23.857459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:23.857459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:23.859274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:23.859274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:23.859274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:38:23.859274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:38:23.859274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:38:23.859274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:38:23.859274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:38:23.859274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:38:23.859274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 30 00:38:23.859274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:38:23.859274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 464/930] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:38:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 30 00:38:23.864664 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c69cb7-b162-4215-8cb8-3b1e5668a039 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:23.986422 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c69cb7-b162-4215-8cb8-3b1e5668a039 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:24.032307 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ed263-ad20-4604-94e2-5f99861ca665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:24.032840 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ed263-ad20-4604-94e2-5f99861ca665 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:24.033133 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ed263-ad20-4604-94e2-5f99861ca665 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:24.033389 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ed263-ad20-4604-94e2-5f99861ca665 None admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:24.034120 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ed263-ad20-4604-94e2-5f99861ca665 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:24.034422 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ed263-ad20-4604-94e2-5f99861ca665 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:24.035852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ed263-ad20-4604-94e2-5f99861ca665 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:24.059605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 467/931] 10.222.0.36 () {56 vars in 1143 bytes} [Sun Oct 30 00:38:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:24.066423 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402db913-4a82-4bac-a21d-22a4679c3a40 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:24.201869 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402db913-4a82-4bac-a21d-22a4679c3a40 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:24.215526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:24.244870 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072196d1-55a9-4a74-ba83-6371d7a260ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:24.245792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072196d1-55a9-4a74-ba83-6371d7a260ec None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:24.246070 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072196d1-55a9-4a74-ba83-6371d7a260ec None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:24.246341 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072196d1-55a9-4a74-ba83-6371d7a260ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:24.327029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072196d1-55a9-4a74-ba83-6371d7a260ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:24.328456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072196d1-55a9-4a74-ba83-6371d7a260ec None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:24.452826 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 465/932] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:38:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 30 00:38:24.458704 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ed263-ad20-4604-94e2-5f99861ca665 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:24.498552 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:24.586514 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ed263-ad20-4604-94e2-5f99861ca665 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:24.625543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aea174-ff80-4357-a4d6-fdb7a7be216b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:24.626318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aea174-ff80-4357-a4d6-fdb7a7be216b None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:24.626429 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aea174-ff80-4357-a4d6-fdb7a7be216b None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:24.626700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aea174-ff80-4357-a4d6-fdb7a7be216b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:24.639121 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aea174-ff80-4357-a4d6-fdb7a7be216b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:24.641032 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aea174-ff80-4357-a4d6-fdb7a7be216b None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:24.661239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-32aea174-ff80-4357-a4d6-fdb7a7be216b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:38:24.661239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:24.661239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:24.661239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:24.661239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:24.661239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:24.661239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:24.661239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:24.661239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:24.661239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:24.661239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:24.661239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:24.661239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:38:24.661239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:38:24.661239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:38:24.661239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:38:24.662411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:24.662411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:24.662411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:24.662411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:24.662411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:24.662411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:24.662411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:24.662411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:24.662411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:24.662411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:24.662411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:24.662411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:24.662411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:24.662411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:24.662411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:24.662411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:24.662411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:38:24.668759 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:38:24.668759 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:38:24.668759 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:24.668759 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 468/933] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:24.669507 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072196d1-55a9-4a74-ba83-6371d7a260ec None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:24.682058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:24.779241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072196d1-55a9-4a74-ba83-6371d7a260ec None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:24.819105 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31e287-7bcc-4427-a105-c9ed97b4fa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:24.819744 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31e287-7bcc-4427-a105-c9ed97b4fa04 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:24.819995 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31e287-7bcc-4427-a105-c9ed97b4fa04 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:24.820297 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31e287-7bcc-4427-a105-c9ed97b4fa04 None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:24.820953 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31e287-7bcc-4427-a105-c9ed97b4fa04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:24.821282 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31e287-7bcc-4427-a105-c9ed97b4fa04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:24.823715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31e287-7bcc-4427-a105-c9ed97b4fa04 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:24.835761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 466/934] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:24.841912 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aea174-ff80-4357-a4d6-fdb7a7be216b None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:24.942633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:24.961238 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aea174-ff80-4357-a4d6-fdb7a7be216b None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:25.003873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03887dbe-8c4d-4d0a-baae-7566cfcc17e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:25.004950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03887dbe-8c4d-4d0a-baae-7566cfcc17e3 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:25.005384 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03887dbe-8c4d-4d0a-baae-7566cfcc17e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:25.005879 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03887dbe-8c4d-4d0a-baae-7566cfcc17e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:25.016131 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03887dbe-8c4d-4d0a-baae-7566cfcc17e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:25.017775 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03887dbe-8c4d-4d0a-baae-7566cfcc17e3 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:25.030415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-03887dbe-8c4d-4d0a-baae-7566cfcc17e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:38:25.030415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:25.030415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:25.030415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:25.030415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:25.030415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:25.030415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:25.030415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:25.030415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:25.030415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:25.030415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:25.030415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:25.030415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:38:25.030415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:38:25.030415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:38:25.031430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:38:25.031430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:25.031430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:25.031430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:25.031430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:25.031430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:25.031430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:25.031430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:25.031430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:25.031430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:25.031430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:25.031430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:25.031430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:25.031430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:25.031430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:25.031430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:25.031430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:25.032793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:38:25.032793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:38:25.032793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:38:25.032793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:38:25.032793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:38:25.032793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:38:25.032793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:38:25.032793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:25.032793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 469/935] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:38:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:25.037916 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31e287-7bcc-4427-a105-c9ed97b4fa04 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:25.128511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:25.184299 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31e287-7bcc-4427-a105-c9ed97b4fa04 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:25.226772 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5217759-9668-41f0-ae56-fdab5c44d2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:25.227467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5217759-9668-41f0-ae56-fdab5c44d2bc None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:25.227842 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5217759-9668-41f0-ae56-fdab5c44d2bc None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:25.228746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5217759-9668-41f0-ae56-fdab5c44d2bc None admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:25.229533 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5217759-9668-41f0-ae56-fdab5c44d2bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:25.229925 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5217759-9668-41f0-ae56-fdab5c44d2bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:25.231353 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5217759-9668-41f0-ae56-fdab5c44d2bc None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:25.247197 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 467/936] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:38:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:25.253549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03887dbe-8c4d-4d0a-baae-7566cfcc17e3 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:25.380140 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03887dbe-8c4d-4d0a-baae-7566cfcc17e3 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:25.435385 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf033a-e979-49cf-b7bf-a5dadb8c09ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:25.436016 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf033a-e979-49cf-b7bf-a5dadb8c09ce None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:25.436389 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf033a-e979-49cf-b7bf-a5dadb8c09ce None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:25.436801 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf033a-e979-49cf-b7bf-a5dadb8c09ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:25.437347 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf033a-e979-49cf-b7bf-a5dadb8c09ce None admin] RBAC: Adding query filter params (role.id=9a8de43206fa4a8fa4901488bf86f9f5, scope.project.id=ab3a0e5fc5a344739b58ec8313639496, user.id=5ded89d48d6c4a15a5c5829f594f8776) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:25.437669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf033a-e979-49cf-b7bf-a5dadb8c09ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:25.439139 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf033a-e979-49cf-b7bf-a5dadb8c09ce None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:25.449812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 470/937] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:38:25 2022] GET /identity/v3/role_assignments?user.id=5ded89d48d6c4a15a5c5829f594f8776&scope.project.id=ab3a0e5fc5a344739b58ec8313639496&role.id=9a8de43206fa4a8fa4901488bf86f9f5 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:27.822607 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838c9c7-5a9a-4e5d-9b3f-d83fae811dc3 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:27.823354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838c9c7-5a9a-4e5d-9b3f-d83fae811dc3 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:27.823354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838c9c7-5a9a-4e5d-9b3f-d83fae811dc3 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:27.824448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 468/938] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:38:27.831499 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da881e4-3961-4019-8b31-df516803afce None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:27.832155 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da881e4-3961-4019-8b31-df516803afce None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:27.832688 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da881e4-3961-4019-8b31-df516803afce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:27.864964 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-6da881e4-3961-4019-8b31-df516803afce None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:38:27.866836 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-6da881e4-3961-4019-8b31-df516803afce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:38:27.870153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:27.973295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 471/939] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:38:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:38:28.292772 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838c9c7-5a9a-4e5d-9b3f-d83fae811dc3 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:28.429890 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838c9c7-5a9a-4e5d-9b3f-d83fae811dc3 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:28.472071 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:28.479278 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc23a98-07ca-4f8b-bb6e-bb634609b912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:28.479925 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc23a98-07ca-4f8b-bb6e-bb634609b912 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:28.480481 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc23a98-07ca-4f8b-bb6e-bb634609b912 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:28.480858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc23a98-07ca-4f8b-bb6e-bb634609b912 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:28.495239 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc23a98-07ca-4f8b-bb6e-bb634609b912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:38:28.501495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-edc23a98-07ca-4f8b-bb6e-bb634609b912 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:38:28.501495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:28.501495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:28.501495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:28.501495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:28.501495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:28.501495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:28.501495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:28.501495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:28.501495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:28.501495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:28.501495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:28.501495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:38:28.501495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:38:28.501495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:38:28.501495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:38:28.502566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:38:28.502566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:38:28.502566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:38:28.502566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:38:28.502566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:28.502566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:28.502566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:28.502566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:28.502566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:28.502566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:28.502566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:28.502566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:28.502566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:28.502566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:28.502566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:28.502566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:28.503495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:28.503495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:28.503495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:28.503495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:28.503495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:38:28.503495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:38:28.503495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:38:28.503495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:38:28.503495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:38:28.503495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:38:28.503495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:38:28.503495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:38:28.503495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 469/940] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:38:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:28.510071 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da881e4-3961-4019-8b31-df516803afce None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:28.633499 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da881e4-3961-4019-8b31-df516803afce None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:28.675113 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74149d-48c0-4664-a8cb-bc5bcb32cc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:28.675798 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74149d-48c0-4664-a8cb-bc5bcb32cc1e None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:28.676076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74149d-48c0-4664-a8cb-bc5bcb32cc1e None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:28.676378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74149d-48c0-4664-a8cb-bc5bcb32cc1e None admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:28.677051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74149d-48c0-4664-a8cb-bc5bcb32cc1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:28.677350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74149d-48c0-4664-a8cb-bc5bcb32cc1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:28.678829 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74149d-48c0-4664-a8cb-bc5bcb32cc1e None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:28.688669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 472/941] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:38:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:28.695077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc23a98-07ca-4f8b-bb6e-bb634609b912 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:28.809690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc23a98-07ca-4f8b-bb6e-bb634609b912 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:28.846108 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205be55-eec3-4f19-96f4-78d558488b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:28.846794 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205be55-eec3-4f19-96f4-78d558488b97 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:28.847010 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205be55-eec3-4f19-96f4-78d558488b97 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:28.847309 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205be55-eec3-4f19-96f4-78d558488b97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:28.858297 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205be55-eec3-4f19-96f4-78d558488b97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:28.859747 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205be55-eec3-4f19-96f4-78d558488b97 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:28.875555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-0205be55-eec3-4f19-96f4-78d558488b97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:38:28.875555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:28.875555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:28.875555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:28.875555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:28.875555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:28.875555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:28.875555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:28.875555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:28.875555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:28.875555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:28.875555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:28.875555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:38:28.875555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:38:28.875555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:38:28.875555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:38:28.881392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:28.881392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:28.881392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:28.881392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:28.881392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:28.881392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:28.881392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:28.881392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:28.881392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:28.881392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:28.881392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:28.881392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:28.881392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:28.881392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:28.881392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:28.881392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:28.881392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:38:28.882539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:38:28.882539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:38:28.882539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:38:28.882539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 470/942] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:38:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:28.882838 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74149d-48c0-4664-a8cb-bc5bcb32cc1e None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:28.885569 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:29.000760 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74149d-48c0-4664-a8cb-bc5bcb32cc1e None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:29.038308 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff0f96-2fef-4816-83ee-380b01a3aa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:29.038923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff0f96-2fef-4816-83ee-380b01a3aa11 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:29.039199 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff0f96-2fef-4816-83ee-380b01a3aa11 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:29.039469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff0f96-2fef-4816-83ee-380b01a3aa11 None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:29.040137 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff0f96-2fef-4816-83ee-380b01a3aa11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:29.040443 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff0f96-2fef-4816-83ee-380b01a3aa11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:29.042144 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff0f96-2fef-4816-83ee-380b01a3aa11 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:29.054995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 473/943] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:38:29.063518 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205be55-eec3-4f19-96f4-78d558488b97 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:29.111683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:29.184534 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205be55-eec3-4f19-96f4-78d558488b97 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:29.226670 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5144187-5ee9-47e6-a26d-87a0acc4488e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:29.227318 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5144187-5ee9-47e6-a26d-87a0acc4488e None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:29.227599 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5144187-5ee9-47e6-a26d-87a0acc4488e None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:29.227884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5144187-5ee9-47e6-a26d-87a0acc4488e None admin] PATH_INFO: `/v3/users/placement` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:29.236423 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5144187-5ee9-47e6-a26d-87a0acc4488e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:29.237865 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5144187-5ee9-47e6-a26d-87a0acc4488e None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:29.251775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-b5144187-5ee9-47e6-a26d-87a0acc4488e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 30 00:38:29.251775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:29.251775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:29.251775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:29.251775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:29.251775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:29.251775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:29.251775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:29.251775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:29.251775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:29.251775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:29.251775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:29.251775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:38:29.251775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:38:29.251775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:38:29.251775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:38:29.255228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:29.255228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:29.255228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:38:29.255228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:38:29.255228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:38:29.255228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:38:29.255228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:29.255228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:29.255228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:29.255228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:29.255228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:29.255228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:29.255228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:29.255228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:29.255228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:29.255228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:29.255228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:29.255228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:29.256614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:29.256614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:29.256614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:38:29.256614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:38:29.256614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:38:29.256614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:38:29.256614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:38:29.256614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:38:29.256614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 30 00:38:29.256614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:38:29.256614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 471/944] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:38:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:29.258898 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff0f96-2fef-4816-83ee-380b01a3aa11 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:29.338960 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:29.383341 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff0f96-2fef-4816-83ee-380b01a3aa11 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:29.426584 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b631e9-da36-4d23-a677-4bd79582a78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:29.427226 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b631e9-da36-4d23-a677-4bd79582a78d None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:29.427577 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b631e9-da36-4d23-a677-4bd79582a78d None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:29.427853 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b631e9-da36-4d23-a677-4bd79582a78d None admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:29.428714 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b631e9-da36-4d23-a677-4bd79582a78d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:29.429023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b631e9-da36-4d23-a677-4bd79582a78d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:29.430574 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b631e9-da36-4d23-a677-4bd79582a78d None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:29.460736 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 474/945] 10.222.0.36 () {56 vars in 1143 bytes} [Sun Oct 30 00:38:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:38:29.466992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5144187-5ee9-47e6-a26d-87a0acc4488e None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:29.586188 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5144187-5ee9-47e6-a26d-87a0acc4488e None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:29.589217 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:29.636369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79385590-f8dc-4aa2-ad78-7d7e678fdf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:29.637396 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79385590-f8dc-4aa2-ad78-7d7e678fdf20 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:29.637862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79385590-f8dc-4aa2-ad78-7d7e678fdf20 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:29.638221 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79385590-f8dc-4aa2-ad78-7d7e678fdf20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:29.651973 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79385590-f8dc-4aa2-ad78-7d7e678fdf20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:29.653700 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79385590-f8dc-4aa2-ad78-7d7e678fdf20 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:29.671033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-79385590-f8dc-4aa2-ad78-7d7e678fdf20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:38:29.671033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:29.671033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:29.671033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:29.671033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:29.671033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:29.671033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:29.671033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:29.671033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:29.671033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:29.671033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:29.671033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:29.671033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:38:29.671033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:38:29.671033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:38:29.671033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:38:29.672376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:29.672376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:29.672376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:29.672376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:29.672376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:29.672376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:29.672376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:29.672376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:29.672376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:29.672376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:29.672376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:29.672376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:29.672376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:29.672376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:29.672376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:29.672376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:29.672376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:38:29.673361 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:38:29.673361 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:38:29.673361 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:38:29.673361 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 472/946] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:29.678350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b631e9-da36-4d23-a677-4bd79582a78d None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:29.787660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b631e9-da36-4d23-a677-4bd79582a78d None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:29.810853 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:29.826493 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1f63e-1012-4425-91ae-352452ce45e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:29.827105 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1f63e-1012-4425-91ae-352452ce45e6 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:29.827387 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1f63e-1012-4425-91ae-352452ce45e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:29.827769 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1f63e-1012-4425-91ae-352452ce45e6 None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:29.828372 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1f63e-1012-4425-91ae-352452ce45e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:29.828678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1f63e-1012-4425-91ae-352452ce45e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:29.830723 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1f63e-1012-4425-91ae-352452ce45e6 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:29.845430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 475/947] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:29.851573 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79385590-f8dc-4aa2-ad78-7d7e678fdf20 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:29.968811 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79385590-f8dc-4aa2-ad78-7d7e678fdf20 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:30.006082 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5eb85-b5bc-414a-98de-486c223d59d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:30.007003 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5eb85-b5bc-414a-98de-486c223d59d3 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:30.007439 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5eb85-b5bc-414a-98de-486c223d59d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:30.007862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5eb85-b5bc-414a-98de-486c223d59d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:30.015760 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5eb85-b5bc-414a-98de-486c223d59d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:30.017320 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5eb85-b5bc-414a-98de-486c223d59d3 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:30.029672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-30f5eb85-b5bc-414a-98de-486c223d59d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:38:30.029672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:30.029672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:30.029672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:30.029672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:30.029672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:30.029672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:30.029672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:30.029672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:30.029672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:30.029672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:30.029672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:30.029672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:38:30.029672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:38:30.029672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:38:30.030841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:38:30.030841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:30.030841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:30.030841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:30.030841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:30.030841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:30.030841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:30.030841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:30.030841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:30.030841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:30.030841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:30.030841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:30.030841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:30.030841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:30.030841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:30.030841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:30.030841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:30.031775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:38:30.031775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:38:30.031775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:38:30.031775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:38:30.031775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:38:30.031775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:38:30.031775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:38:30.031775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:38:30.031775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 473/948] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:38:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:30.037422 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1f63e-1012-4425-91ae-352452ce45e6 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:30.152946 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1f63e-1012-4425-91ae-352452ce45e6 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:30.195027 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1bf37-6b14-4576-bc1e-adf9414dec6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:30.195641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1bf37-6b14-4576-bc1e-adf9414dec6d None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:30.195920 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1bf37-6b14-4576-bc1e-adf9414dec6d None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:30.196223 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1bf37-6b14-4576-bc1e-adf9414dec6d None admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:30.197555 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1bf37-6b14-4576-bc1e-adf9414dec6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:30.197777 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1bf37-6b14-4576-bc1e-adf9414dec6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:30.199327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1bf37-6b14-4576-bc1e-adf9414dec6d None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:30.212895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 476/949] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:38:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:30.219342 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5eb85-b5bc-414a-98de-486c223d59d3 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:30.236272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:30.330748 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5eb85-b5bc-414a-98de-486c223d59d3 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:30.373550 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41391d-2f0e-4d5d-a266-4f49294ce2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:30.374224 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41391d-2f0e-4d5d-a266-4f49294ce2ba None admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:30.374527 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41391d-2f0e-4d5d-a266-4f49294ce2ba None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:30.374860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41391d-2f0e-4d5d-a266-4f49294ce2ba None admin] PATH_INFO: `/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/5ded89d48d6c4a15a5c5829f594f8776/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:30.377525 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41391d-2f0e-4d5d-a266-4f49294ce2ba None admin] RBAC: Authorizing `identity:create_grant(project_id=ab3a0e5fc5a344739b58ec8313639496, user_id=5ded89d48d6c4a15a5c5829f594f8776, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:30.379072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41391d-2f0e-4d5d-a266-4f49294ce2ba None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:30.396151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 474/950] 10.222.0.36 () {58 vars in 1439 bytes} [Sun Oct 30 00:38:30 2022] PUT /identity/v3/projects/ab3a0e5fc5a344739b58ec8313639496/users/5ded89d48d6c4a15a5c5829f594f8776/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:38:32.738237 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb44ba-b720-4a23-925e-b773a585431c None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:32.738794 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb44ba-b720-4a23-925e-b773a585431c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:32.738911 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb44ba-b720-4a23-925e-b773a585431c None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:32.740442 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 477/951] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:38:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:38:32.748108 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aad4ae-ba64-40e2-b544-044d389bdd14 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:32.748108 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aad4ae-ba64-40e2-b544-044d389bdd14 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:32.748108 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aad4ae-ba64-40e2-b544-044d389bdd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:32.768004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-f9aad4ae-ba64-40e2-b544-044d389bdd14 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:38:32.769348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f9aad4ae-ba64-40e2-b544-044d389bdd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:38:32.850339 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 475/952] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:38:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:38:33.162770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb44ba-b720-4a23-925e-b773a585431c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:33.214294 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:33.286361 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb44ba-b720-4a23-925e-b773a585431c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:33.328837 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ace6b8-17ea-40fd-824a-8d57f6f5c47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:33.329469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ace6b8-17ea-40fd-824a-8d57f6f5c47d None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:33.329750 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ace6b8-17ea-40fd-824a-8d57f6f5c47d None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:33.330025 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ace6b8-17ea-40fd-824a-8d57f6f5c47d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:33.340049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ace6b8-17ea-40fd-824a-8d57f6f5c47d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 30 00:38:33.344805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-53ace6b8-17ea-40fd-824a-8d57f6f5c47d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:38:33.344805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:33.344805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:33.344805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:33.344805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:33.344805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:33.344805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:33.344805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:33.344805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:33.344805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:33.344805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:33.344805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:33.344805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 30 00:38:33.344805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 30 00:38:33.344805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 30 00:38:33.344805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 30 00:38:33.345781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 30 00:38:33.345781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 30 00:38:33.345781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 30 00:38:33.345781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 30 00:38:33.345781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:33.345781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:33.345781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:33.345781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:33.345781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:33.345781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:33.345781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:33.345781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:33.345781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:33.345781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:33.345781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:33.345781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:33.346626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:33.346626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:33.346626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:33.346626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:33.346626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 30 00:38:33.346626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 30 00:38:33.346626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 30 00:38:33.346626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 30 00:38:33.346626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 30 00:38:33.346626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 30 00:38:33.346626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 30 00:38:33.346626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:33.346626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 478/953] 10.222.0.36 () {56 vars in 1096 bytes} [Sun Oct 30 00:38:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:33.351972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aad4ae-ba64-40e2-b544-044d389bdd14 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:33.356156 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:33.477088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aad4ae-ba64-40e2-b544-044d389bdd14 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:33.518931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77d869-847c-4514-8fa4-e90fcaa48e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:33.519592 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77d869-847c-4514-8fa4-e90fcaa48e0b None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:33.519928 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77d869-847c-4514-8fa4-e90fcaa48e0b None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:33.520250 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77d869-847c-4514-8fa4-e90fcaa48e0b None admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:33.521117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77d869-847c-4514-8fa4-e90fcaa48e0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:33.521425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77d869-847c-4514-8fa4-e90fcaa48e0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:33.522981 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77d869-847c-4514-8fa4-e90fcaa48e0b None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:33.533938 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 476/954] 10.222.0.36 () {56 vars in 1099 bytes} [Sun Oct 30 00:38:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:33.540375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ace6b8-17ea-40fd-824a-8d57f6f5c47d None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:33.661969 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:33.664905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ace6b8-17ea-40fd-824a-8d57f6f5c47d None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:33.706574 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba5f2e-3264-4a59-bcc0-26f4467e0b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:33.707221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba5f2e-3264-4a59-bcc0-26f4467e0b8e None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:33.707503 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba5f2e-3264-4a59-bcc0-26f4467e0b8e None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:33.707774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba5f2e-3264-4a59-bcc0-26f4467e0b8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:33.720579 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba5f2e-3264-4a59-bcc0-26f4467e0b8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:33.722122 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba5f2e-3264-4a59-bcc0-26f4467e0b8e None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:33.739763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-77ba5f2e-3264-4a59-bcc0-26f4467e0b8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:38:33.739763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:33.739763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:33.739763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:33.739763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:33.739763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:33.739763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:33.739763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:33.739763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:33.739763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:33.739763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:33.739763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:33.739763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:38:33.739763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:38:33.739763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:38:33.739763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:38:33.741354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:33.741354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:33.741354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:33.741354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:33.741354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:33.741354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:33.741354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:33.741354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:33.741354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:33.741354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:33.741354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:33.741354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:33.741354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:33.741354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:33.741354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:33.741354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:33.741354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:38:33.742292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:38:33.742292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:38:33.742292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:33.742292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 479/955] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:33.747698 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77d869-847c-4514-8fa4-e90fcaa48e0b None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:33.840653 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:33.884458 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77d869-847c-4514-8fa4-e90fcaa48e0b None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:33.926251 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e84db-a0a1-4d98-80f2-b6c522f7552a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:33.926884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e84db-a0a1-4d98-80f2-b6c522f7552a None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:33.929751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e84db-a0a1-4d98-80f2-b6c522f7552a None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:33.929751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e84db-a0a1-4d98-80f2-b6c522f7552a None admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:33.929751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e84db-a0a1-4d98-80f2-b6c522f7552a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:33.929751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e84db-a0a1-4d98-80f2-b6c522f7552a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:33.929751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e84db-a0a1-4d98-80f2-b6c522f7552a None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:33.944266 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 477/956] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:33.950231 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba5f2e-3264-4a59-bcc0-26f4467e0b8e None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:34.082665 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba5f2e-3264-4a59-bcc0-26f4467e0b8e None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:34.136288 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059abc6-10c2-48a6-b640-76668e7575cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:34.136930 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059abc6-10c2-48a6-b640-76668e7575cd None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:34.137214 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059abc6-10c2-48a6-b640-76668e7575cd None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:34.137505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059abc6-10c2-48a6-b640-76668e7575cd None admin] PATH_INFO: `/v3/users/placement` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:34.147471 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059abc6-10c2-48a6-b640-76668e7575cd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:34.147962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059abc6-10c2-48a6-b640-76668e7575cd None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:34.161641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-b059abc6-10c2-48a6-b640-76668e7575cd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 30 00:38:34.161641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:34.161641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:34.161641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:34.161641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:34.161641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:34.161641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:34.161641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:34.161641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:34.161641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:34.161641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:34.161641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:34.161641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 30 00:38:34.161641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 30 00:38:34.161641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 30 00:38:34.161641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 30 00:38:34.163212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:34.163212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:34.163212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 30 00:38:34.163212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:38:34.163212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 30 00:38:34.163212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 30 00:38:34.163212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:34.163212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:34.163212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:34.163212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:34.163212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:34.163212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:34.163212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:34.163212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:34.163212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:34.163212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:34.163212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:34.163212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:34.164542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:34.164542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:34.164542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 30 00:38:34.164542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 30 00:38:34.164542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 30 00:38:34.164542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 30 00:38:34.164542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 30 00:38:34.164542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 30 00:38:34.164542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 30 00:38:34.164542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:34.164542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 480/957] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:38:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:34.169851 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e84db-a0a1-4d98-80f2-b6c522f7552a None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:34.275610 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e84db-a0a1-4d98-80f2-b6c522f7552a None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:34.305298 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:34.316259 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9f375-b40a-4492-8779-d58556214d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:34.316872 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9f375-b40a-4492-8779-d58556214d92 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:34.317213 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9f375-b40a-4492-8779-d58556214d92 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:34.317490 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9f375-b40a-4492-8779-d58556214d92 None admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:34.318215 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9f375-b40a-4492-8779-d58556214d92 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:34.318520 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9f375-b40a-4492-8779-d58556214d92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:34.319938 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9f375-b40a-4492-8779-d58556214d92 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:34.339126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 478/958] 10.222.0.36 () {56 vars in 1143 bytes} [Sun Oct 30 00:38:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:34.348369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059abc6-10c2-48a6-b640-76668e7575cd None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:34.370373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:34.479399 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059abc6-10c2-48a6-b640-76668e7575cd None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:34.516575 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb203346-0af3-470a-b91a-43deeb8248fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:34.517190 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb203346-0af3-470a-b91a-43deeb8248fa None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:34.517475 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb203346-0af3-470a-b91a-43deeb8248fa None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:34.517750 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb203346-0af3-470a-b91a-43deeb8248fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:34.594755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb203346-0af3-470a-b91a-43deeb8248fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:34.596129 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb203346-0af3-470a-b91a-43deeb8248fa None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:34.611155 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:34.703740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 481/959] 10.222.0.36 () {58 vars in 1282 bytes} [Sun Oct 30 00:38:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 30 00:38:34.709826 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9f375-b40a-4492-8779-d58556214d92 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:34.825112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9f375-b40a-4492-8779-d58556214d92 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:34.862897 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b44f23-896d-4002-adc0-44750ca4a20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:34.863564 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b44f23-896d-4002-adc0-44750ca4a20c None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:34.863754 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b44f23-896d-4002-adc0-44750ca4a20c None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:34.864026 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b44f23-896d-4002-adc0-44750ca4a20c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:34.880169 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b44f23-896d-4002-adc0-44750ca4a20c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:34.881557 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b44f23-896d-4002-adc0-44750ca4a20c None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:34.902459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-25b44f23-896d-4002-adc0-44750ca4a20c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:38:34.902459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:34.902459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:34.902459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:34.902459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:34.902459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:34.902459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:34.902459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:34.902459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:34.902459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:34.902459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:34.902459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:34.902459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 30 00:38:34.902459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 30 00:38:34.902459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 30 00:38:34.902459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 30 00:38:34.903569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:34.903569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:34.903569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:34.903569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:34.903569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:34.903569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:34.903569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:34.903569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:34.903569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:34.903569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:34.903569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:34.903569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:34.903569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:34.903569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:34.903569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:34.903569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:34.903569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 30 00:38:34.904861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 30 00:38:34.904861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 30 00:38:34.904861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:38:34.904861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 479/960] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:38:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:34.909599 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb203346-0af3-470a-b91a-43deeb8248fa None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:35.028712 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb203346-0af3-470a-b91a-43deeb8248fa None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:35.043778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:35.069820 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacee8fe-8959-420f-9583-5647edc5b76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:35.070532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacee8fe-8959-420f-9583-5647edc5b76d None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:35.070814 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacee8fe-8959-420f-9583-5647edc5b76d None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:35.071089 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacee8fe-8959-420f-9583-5647edc5b76d None admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:35.071804 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacee8fe-8959-420f-9583-5647edc5b76d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:35.072158 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacee8fe-8959-420f-9583-5647edc5b76d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:35.073578 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacee8fe-8959-420f-9583-5647edc5b76d None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:35.087177 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 482/961] 10.222.0.36 () {56 vars in 1109 bytes} [Sun Oct 30 00:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:35.094045 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b44f23-896d-4002-adc0-44750ca4a20c None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:35.136261 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:35.212985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b44f23-896d-4002-adc0-44750ca4a20c None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:35.253171 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3777c7-00de-48c3-ba65-cb47569577ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:35.253839 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3777c7-00de-48c3-ba65-cb47569577ab None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:35.254122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3777c7-00de-48c3-ba65-cb47569577ab None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:35.254399 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3777c7-00de-48c3-ba65-cb47569577ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:35.262975 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3777c7-00de-48c3-ba65-cb47569577ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:35.264628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3777c7-00de-48c3-ba65-cb47569577ab None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:35.278687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application [None req-4b3777c7-00de-48c3-ba65-cb47569577ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:38:35.278687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:35.278687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:35.278687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:35.278687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:35.278687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:35.278687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:35.278687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:35.278687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:35.278687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:35.278687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:35.278687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:35.278687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 30 00:38:35.278687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 30 00:38:35.278687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 30 00:38:35.279707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 30 00:38:35.279707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:35.279707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:35.279707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:35.279707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:35.279707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:35.279707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:35.279707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:35.279707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:35.279707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:35.279707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:35.279707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:35.279707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:35.279707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:35.279707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:35.279707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:35.279707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:35.281516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 30 00:38:35.281516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 30 00:38:35.281516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 30 00:38:35.281516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 30 00:38:35.281516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 30 00:38:35.281516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 30 00:38:35.281516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 30 00:38:35.281516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: ERROR keystone.server.flask.application Oct 30 00:38:35.281516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 480/962] 10.222.0.36 () {56 vars in 1111 bytes} [Sun Oct 30 00:38:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:35.286176 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacee8fe-8959-420f-9583-5647edc5b76d None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:35.403671 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacee8fe-8959-420f-9583-5647edc5b76d None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:35.451104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9a98b-0b4b-45dd-87c2-f5600734960f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:35.451748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9a98b-0b4b-45dd-87c2-f5600734960f None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:35.452034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9a98b-0b4b-45dd-87c2-f5600734960f None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:35.452350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9a98b-0b4b-45dd-87c2-f5600734960f None admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:35.453067 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9a98b-0b4b-45dd-87c2-f5600734960f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:35.453376 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9a98b-0b4b-45dd-87c2-f5600734960f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:35.454878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9a98b-0b4b-45dd-87c2-f5600734960f None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:35.470218 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 483/963] 10.222.0.36 () {56 vars in 1148 bytes} [Sun Oct 30 00:38:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:35.476908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3777c7-00de-48c3-ba65-cb47569577ab None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:35.582009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:35.589286 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3777c7-00de-48c3-ba65-cb47569577ab None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:35.625109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685bc35-3c91-453c-aabe-87e85349aff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:35.625726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685bc35-3c91-453c-aabe-87e85349aff4 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:35.626035 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685bc35-3c91-453c-aabe-87e85349aff4 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:35.626315 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685bc35-3c91-453c-aabe-87e85349aff4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:35.627052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685bc35-3c91-453c-aabe-87e85349aff4 None admin] RBAC: Adding query filter params (role.id=980241d45fdb435d9f32c434057c71e4, scope.project.id=ab3a0e5fc5a344739b58ec8313639496, user.id=5ded89d48d6c4a15a5c5829f594f8776) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:35.627362 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685bc35-3c91-453c-aabe-87e85349aff4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:35.628843 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685bc35-3c91-453c-aabe-87e85349aff4 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:35.635768 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 481/964] 10.222.0.36 () {56 vars in 1374 bytes} [Sun Oct 30 00:38:35 2022] GET /identity/v3/role_assignments?user.id=5ded89d48d6c4a15a5c5829f594f8776&scope.project.id=ab3a0e5fc5a344739b58ec8313639496&role.id=980241d45fdb435d9f32c434057c71e4 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:37.850023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4db8d-2fb9-4148-9848-e541659e5d7c None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:37.850572 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4db8d-2fb9-4148-9848-e541659e5d7c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:37.850572 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4db8d-2fb9-4148-9848-e541659e5d7c None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:37.851906 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 484/965] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:38:37.858199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86df89b-6a74-4ef8-8c9d-9ecb541c87bd None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:37.858703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86df89b-6a74-4ef8-8c9d-9ecb541c87bd None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:37.858788 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86df89b-6a74-4ef8-8c9d-9ecb541c87bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:37.879545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-d86df89b-6a74-4ef8-8c9d-9ecb541c87bd None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:38:37.880823 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d86df89b-6a74-4ef8-8c9d-9ecb541c87bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:38:37.951723 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 482/966] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:38:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:38:38.224793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4db8d-2fb9-4148-9848-e541659e5d7c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:38.256304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:38.332771 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4db8d-2fb9-4148-9848-e541659e5d7c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:38.370450 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d57eb-129f-46f2-a355-708ba4f83f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:38.371088 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d57eb-129f-46f2-a355-708ba4f83f8a None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:38.371367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d57eb-129f-46f2-a355-708ba4f83f8a None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:38.371638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d57eb-129f-46f2-a355-708ba4f83f8a None admin] PATH_INFO: `/v3/services/placement` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:38.372253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d57eb-129f-46f2-a355-708ba4f83f8a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:38.373627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d57eb-129f-46f2-a355-708ba4f83f8a None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:38.382982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-d97d57eb-129f-46f2-a355-708ba4f83f8a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 30 00:38:38.382982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:38.382982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:38:38.382982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:38:38.382982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:38:38.382982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:38:38.382982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:38:38.382982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:38:38.382982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 30 00:38:38.382982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:38.382982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:38:38.382982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:38.382982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:38.382982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:38.382982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:38.382982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:38.382982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:38.384727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:38.384727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:38.384727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:38.384727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:38.384727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:38.384727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:38.384727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:38:38.384727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:38:38.384727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:38:38.384727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:38:38.384727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:38.384727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:38.384727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:38.384727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:38.384727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:38.384727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:38.385821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:38.385821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:38.385821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:38.385821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:38.385821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:38.385821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:38.385821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:38.385821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:38.385821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:38.385821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:38.385821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:38:38.385821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:38:38.385821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 30 00:38:38.385821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:38.385821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 485/967] 10.222.0.36 () {56 vars in 1117 bytes} [Sun Oct 30 00:38:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:38.390825 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86df89b-6a74-4ef8-8c9d-9ecb541c87bd None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:38.490732 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:38.501783 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86df89b-6a74-4ef8-8c9d-9ecb541c87bd None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:38.537575 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1c44a-4b91-4c0e-b420-486b9916fc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:38.538194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1c44a-4b91-4c0e-b420-486b9916fc12 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:38.538473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1c44a-4b91-4c0e-b420-486b9916fc12 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:38.538744 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1c44a-4b91-4c0e-b420-486b9916fc12 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:38.539374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1c44a-4b91-4c0e-b420-486b9916fc12 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:38.539670 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1c44a-4b91-4c0e-b420-486b9916fc12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:38.541661 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1c44a-4b91-4c0e-b420-486b9916fc12 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:38.547310 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 483/968] 10.222.0.36 () {56 vars in 1116 bytes} [Sun Oct 30 00:38:38 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:38.553248 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d57eb-129f-46f2-a355-708ba4f83f8a None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:38.654403 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:38.660548 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d57eb-129f-46f2-a355-708ba4f83f8a None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:38.696515 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c694c-e2f3-4955-82dd-ed41a35539d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:38.697180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c694c-e2f3-4955-82dd-ed41a35539d3 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:38.697456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c694c-e2f3-4955-82dd-ed41a35539d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:38.697910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c694c-e2f3-4955-82dd-ed41a35539d3 None admin] PATH_INFO: `/v3/services` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:38.698537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c694c-e2f3-4955-82dd-ed41a35539d3 None admin] RBAC: Adding query filter params (type=placement) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:38.698838 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c694c-e2f3-4955-82dd-ed41a35539d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:38.700243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c694c-e2f3-4955-82dd-ed41a35539d3 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:38.705036 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 486/969] 10.222.0.36 () {56 vars in 1116 bytes} [Sun Oct 30 00:38:38 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:40.919668 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7c3ca-da24-486b-bcac-08ea955cfb91 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:40.920203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7c3ca-da24-486b-bcac-08ea955cfb91 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:40.920316 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7c3ca-da24-486b-bcac-08ea955cfb91 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:40.921570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 484/970] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:38:40.926804 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab083a3d-70ab-47c7-b322-7b4977f90364 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:40.927215 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab083a3d-70ab-47c7-b322-7b4977f90364 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:40.927385 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab083a3d-70ab-47c7-b322-7b4977f90364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:40.945292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-ab083a3d-70ab-47c7-b322-7b4977f90364 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:38:40.946467 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-ab083a3d-70ab-47c7-b322-7b4977f90364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:38:41.011574 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 487/971] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:38:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:38:41.282109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7c3ca-da24-486b-bcac-08ea955cfb91 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:41.383740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7c3ca-da24-486b-bcac-08ea955cfb91 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:41.427006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727516ce-7fe3-47aa-b643-d300dbab7fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:41.427623 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727516ce-7fe3-47aa-b643-d300dbab7fb0 None admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:41.427928 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727516ce-7fe3-47aa-b643-d300dbab7fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:41.428252 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727516ce-7fe3-47aa-b643-d300dbab7fb0 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:41.428908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727516ce-7fe3-47aa-b643-d300dbab7fb0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:41.430281 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727516ce-7fe3-47aa-b643-d300dbab7fb0 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:41.448726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 485/972] 10.222.0.36 () {60 vars in 1141 bytes} [Sun Oct 30 00:38:41 2022] POST /identity/v3/services => generated 246 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:38:43.699804 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c95c1f-c473-4ad0-b0f0-066743719bf8 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:43.700377 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c95c1f-c473-4ad0-b0f0-066743719bf8 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:43.700377 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c95c1f-c473-4ad0-b0f0-066743719bf8 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:43.701733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 488/973] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:38:43.707128 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356de5bb-0fc4-47b0-80fa-4d46f06f954e None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:43.707564 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356de5bb-0fc4-47b0-80fa-4d46f06f954e None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:43.707708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356de5bb-0fc4-47b0-80fa-4d46f06f954e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:43.726140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-356de5bb-0fc4-47b0-80fa-4d46f06f954e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:38:43.727326 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-356de5bb-0fc4-47b0-80fa-4d46f06f954e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:38:43.736427 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:43.800588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 486/974] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:38:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:38:44.067955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c95c1f-c473-4ad0-b0f0-066743719bf8 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:44.157285 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:44.167692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c95c1f-c473-4ad0-b0f0-066743719bf8 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:44.202221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba4962-8d11-472d-985e-3d4159a8da2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:44.202831 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba4962-8d11-472d-985e-3d4159a8da2a None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:44.203111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba4962-8d11-472d-985e-3d4159a8da2a None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:44.203382 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba4962-8d11-472d-985e-3d4159a8da2a None admin] PATH_INFO: `/v3/services/placement` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:44.203952 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba4962-8d11-472d-985e-3d4159a8da2a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:44.205347 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba4962-8d11-472d-985e-3d4159a8da2a None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:44.213834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-dbba4962-8d11-472d-985e-3d4159a8da2a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 30 00:38:44.213834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:44.213834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:38:44.213834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:38:44.213834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:38:44.213834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:38:44.213834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:38:44.213834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:38:44.213834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 30 00:38:44.213834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:44.213834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:38:44.213834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:44.213834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:44.213834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:44.213834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:44.213834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:44.213834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:44.214989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:44.214989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:44.214989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:44.214989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:44.214989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:44.214989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:44.214989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:38:44.214989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:38:44.214989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:38:44.214989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:38:44.214989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:44.214989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:44.214989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:44.214989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:44.214989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:44.214989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:44.215834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:44.215834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:44.215834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:44.215834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:44.215834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:44.215834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:44.215834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:44.215834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:44.215834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:44.215834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:44.215834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:38:44.215834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:38:44.215834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 30 00:38:44.215834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:44.215834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 489/975] 10.222.0.36 () {56 vars in 1117 bytes} [Sun Oct 30 00:38:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:44.220961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356de5bb-0fc4-47b0-80fa-4d46f06f954e None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:44.318115 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356de5bb-0fc4-47b0-80fa-4d46f06f954e None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:44.350837 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5eb86-0544-4ddf-bac5-a69b710a8541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:44.351460 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5eb86-0544-4ddf-bac5-a69b710a8541 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:44.351739 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5eb86-0544-4ddf-bac5-a69b710a8541 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:44.352009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5eb86-0544-4ddf-bac5-a69b710a8541 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:44.352654 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e5eb86-0544-4ddf-bac5-a69b710a8541 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:44.353462 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e5eb86-0544-4ddf-bac5-a69b710a8541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:44.354813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e5eb86-0544-4ddf-bac5-a69b710a8541 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:44.359796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 487/976] 10.222.0.36 () {56 vars in 1116 bytes} [Sun Oct 30 00:38:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:44.365660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba4962-8d11-472d-985e-3d4159a8da2a None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:44.479292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba4962-8d11-472d-985e-3d4159a8da2a None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:44.512906 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80919bd4-7364-4390-80a8-382e9dbb00dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:44.513513 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80919bd4-7364-4390-80a8-382e9dbb00dd None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:44.513793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80919bd4-7364-4390-80a8-382e9dbb00dd None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:44.514065 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80919bd4-7364-4390-80a8-382e9dbb00dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:44.514758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80919bd4-7364-4390-80a8-382e9dbb00dd None admin] RBAC: Adding query filter params (interface=public, service_id=dd82af71927e464292f88faf848f016c, region_id=RegionOne) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:44.515065 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80919bd4-7364-4390-80a8-382e9dbb00dd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:44.516467 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80919bd4-7364-4390-80a8-382e9dbb00dd None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:44.521314 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 490/977] 10.222.0.36 () {56 vars in 1251 bytes} [Sun Oct 30 00:38:44 2022] GET /identity/v3/endpoints?service_id=dd82af71927e464292f88faf848f016c&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:44.527251 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5eb86-0544-4ddf-bac5-a69b710a8541 None admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:44.530986 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:44.625266 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5eb86-0544-4ddf-bac5-a69b710a8541 None admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:44.657798 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ba49a-d205-42ad-bc99-0da71f231738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:44.658394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ba49a-d205-42ad-bc99-0da71f231738 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:44.658670 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ba49a-d205-42ad-bc99-0da71f231738 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:44.658942 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ba49a-d205-42ad-bc99-0da71f231738 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:44.659532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ba49a-d205-42ad-bc99-0da71f231738 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:44.660910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ba49a-d205-42ad-bc99-0da71f231738 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:44.667622 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 488/978] 10.222.0.36 () {56 vars in 1087 bytes} [Sun Oct 30 00:38:44 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:38:46.819893 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b70bd9-bca0-4365-9473-1e99a8d24c9f None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:46.820487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b70bd9-bca0-4365-9473-1e99a8d24c9f None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:46.820487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b70bd9-bca0-4365-9473-1e99a8d24c9f None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:46.822363 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 491/979] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:38:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:38:46.827754 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6ea44-c27e-41f9-917e-a9fb8c17140d None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:46.828176 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6ea44-c27e-41f9-917e-a9fb8c17140d None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:46.828339 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6ea44-c27e-41f9-917e-a9fb8c17140d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:46.847372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-a3f6ea44-c27e-41f9-917e-a9fb8c17140d None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:38:46.848598 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a3f6ea44-c27e-41f9-917e-a9fb8c17140d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:38:46.909352 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:46.915507 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 489/980] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:38:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 30 00:38:47.186189 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b70bd9-bca0-4365-9473-1e99a8d24c9f None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:47.205168 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:47.286694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b70bd9-bca0-4365-9473-1e99a8d24c9f None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:47.319566 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478920e-c661-4a34-878e-2a254c1b2f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:47.320206 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478920e-c661-4a34-878e-2a254c1b2f04 None admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:47.320485 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478920e-c661-4a34-878e-2a254c1b2f04 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:47.320758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478920e-c661-4a34-878e-2a254c1b2f04 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:47.321328 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478920e-c661-4a34-878e-2a254c1b2f04 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:47.323253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478920e-c661-4a34-878e-2a254c1b2f04 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:47.331753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application [None req-b478920e-c661-4a34-878e-2a254c1b2f04 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 30 00:38:47.331753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:47.331753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 30 00:38:47.331753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 30 00:38:47.331753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 30 00:38:47.331753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 30 00:38:47.331753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 30 00:38:47.331753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:38:47.331753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 30 00:38:47.331753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:47.331753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 30 00:38:47.331753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:47.331753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 30 00:38:47.331753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 30 00:38:47.331753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 30 00:38:47.331753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 30 00:38:47.331753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 30 00:38:47.332968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 30 00:38:47.332968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 30 00:38:47.332968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 30 00:38:47.332968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 30 00:38:47.332968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 30 00:38:47.332968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 30 00:38:47.332968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 30 00:38:47.332968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 30 00:38:47.332968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 30 00:38:47.332968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 30 00:38:47.332968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 30 00:38:47.332968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 30 00:38:47.332968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 30 00:38:47.332968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 30 00:38:47.332968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 30 00:38:47.332968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.get_or_create( Oct 30 00:38:47.334111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 30 00:38:47.334111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application with Lock( Oct 30 00:38:47.334111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 30 00:38:47.334111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self._enter() Oct 30 00:38:47.334111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 30 00:38:47.334111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 30 00:38:47.334111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 30 00:38:47.334111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application return self.creator() Oct 30 00:38:47.334111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 30 00:38:47.334111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application created_value = creator( Oct 30 00:38:47.334111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 30 00:38:47.334111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 30 00:38:47.334111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 30 00:38:47.334111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: ERROR keystone.server.flask.application Oct 30 00:38:47.334111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 492/981] 10.222.0.36 () {56 vars in 1117 bytes} [Sun Oct 30 00:38:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 30 00:38:47.339526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6ea44-c27e-41f9-917e-a9fb8c17140d None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:47.447949 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6ea44-c27e-41f9-917e-a9fb8c17140d None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:47.482409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a36a8-6b83-4d56-807b-2ce00c904300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:47.483009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a36a8-6b83-4d56-807b-2ce00c904300 None admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:47.483287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a36a8-6b83-4d56-807b-2ce00c904300 None admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:47.483560 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a36a8-6b83-4d56-807b-2ce00c904300 None admin] PATH_INFO: `/v3/services` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:47.484218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a36a8-6b83-4d56-807b-2ce00c904300 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:38:47.484523 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a36a8-6b83-4d56-807b-2ce00c904300 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:47.485882 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a36a8-6b83-4d56-807b-2ce00c904300 None admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:47.490892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 490/982] 10.222.0.36 () {56 vars in 1116 bytes} [Sun Oct 30 00:38:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:38:47.497085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478920e-c661-4a34-878e-2a254c1b2f04 None admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:38:47.576729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:47.595097 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478920e-c661-4a34-878e-2a254c1b2f04 None admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:38:47.627683 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726db326-69e2-4fd4-ada0-577f1a59de34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:38:47.628304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726db326-69e2-4fd4-ada0-577f1a59de34 None admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:38:47.628587 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726db326-69e2-4fd4-ada0-577f1a59de34 None admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:38:47.628859 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726db326-69e2-4fd4-ada0-577f1a59de34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:38:47.629524 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726db326-69e2-4fd4-ada0-577f1a59de34 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:38:47.630886 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726db326-69e2-4fd4-ada0-577f1a59de34 None admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:38:47.649885 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 493/983] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:38:47 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:38:57.487909 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=69 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:38:57.645485 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:39:52.458177 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb060f-d5fd-42cd-baca-ff77eb00c982 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:39:52.459067 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb060f-d5fd-42cd-baca-ff77eb00c982 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:39:52.459520 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb060f-d5fd-42cd-baca-ff77eb00c982 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:39:52.461058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 491/984] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:39:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:39:52.467140 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ed218-f36d-41a0-b347-687c3eb03952 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:39:52.467611 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ed218-f36d-41a0-b347-687c3eb03952 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:39:52.467899 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ed218-f36d-41a0-b347-687c3eb03952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:39:52.611832 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-121ed218-f36d-41a0-b347-687c3eb03952 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:39:52.613174 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-121ed218-f36d-41a0-b347-687c3eb03952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:39:52.841747 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 494/985] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:39:53.177069 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a668946-efcf-43ba-b8e7-569c437acff9 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:39:53.177552 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a668946-efcf-43ba-b8e7-569c437acff9 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:39:53.177933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a668946-efcf-43ba-b8e7-569c437acff9 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:39:53.180039 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 492/986] 10.222.0.36 () {54 vars in 965 bytes} [Sun Oct 30 00:39:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:39:53.186777 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8d4dd-d9ce-4046-a1d1-3a4d84660abc None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:39:53.187062 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8d4dd-d9ce-4046-a1d1-3a4d84660abc None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:39:53.187345 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8d4dd-d9ce-4046-a1d1-3a4d84660abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:39:53.318146 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-d4e8d4dd-d9ce-4046-a1d1-3a4d84660abc None None] MFA Rules not processed for user `fb39b6cc97a1456bb1312f49118c6e10`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:39:53.319731 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d4e8d4dd-d9ce-4046-a1d1-3a4d84660abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:39:53.361214 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:39:53.458553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 495/987] 10.222.0.36 () {58 vars in 1062 bytes} [Sun Oct 30 00:39:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:39:53.459441 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a668946-efcf-43ba-b8e7-569c437acff9 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:39:53.461343 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-2a668946-efcf-43ba-b8e7-569c437acff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:39:53.594657 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a668946-efcf-43ba-b8e7-569c437acff9 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:39:53.682463 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcaa3e9-492c-4488-b576-fe465636be65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:39:53.682734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcaa3e9-492c-4488-b576-fe465636be65 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:39:53.682902 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcaa3e9-492c-4488-b576-fe465636be65 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:39:53.683265 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcaa3e9-492c-4488-b576-fe465636be65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:39:53.685045 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-6bcaa3e9-492c-4488-b576-fe465636be65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:39:53.745729 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:39:53.807314 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcaa3e9-492c-4488-b576-fe465636be65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:39:53.809002 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcaa3e9-492c-4488-b576-fe465636be65 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:39:53.832148 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 493/988] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:39:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:39:56.360026 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5aa50-a2d1-4d6b-8ac9-41def0a6e7ab None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:39:56.360622 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5aa50-a2d1-4d6b-8ac9-41def0a6e7ab None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:39:56.360622 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5aa50-a2d1-4d6b-8ac9-41def0a6e7ab None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:39:56.361951 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 496/989] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:39:56.367270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f351c-fe04-45e3-9093-5e2e97e54b99 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:39:56.367693 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f351c-fe04-45e3-9093-5e2e97e54b99 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:39:56.367803 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f351c-fe04-45e3-9093-5e2e97e54b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:39:56.454378 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-581f351c-fe04-45e3-9093-5e2e97e54b99 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:39:56.455604 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-581f351c-fe04-45e3-9093-5e2e97e54b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:39:56.552091 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 494/990] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:39:56.837733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae5aa50-a2d1-4d6b-8ac9-41def0a6e7ab None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:39:56.839401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-bae5aa50-a2d1-4d6b-8ac9-41def0a6e7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:39:56.943299 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae5aa50-a2d1-4d6b-8ac9-41def0a6e7ab None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:39:57.014605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:39:57.016944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba5db8-de19-40c5-933a-3ac994f014e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:39:57.017752 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba5db8-de19-40c5-933a-3ac994f014e6 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:39:57.018202 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba5db8-de19-40c5-933a-3ac994f014e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:39:57.018658 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba5db8-de19-40c5-933a-3ac994f014e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:39:57.020277 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-06ba5db8-de19-40c5-933a-3ac994f014e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:39:57.127870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba5db8-de19-40c5-933a-3ac994f014e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:39:57.129261 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba5db8-de19-40c5-933a-3ac994f014e6 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:39:57.147698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 497/991] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:39:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:39:59.653947 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a0fd6-b2e8-4a21-b36c-105b6ef2bcc5 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:39:59.654613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a0fd6-b2e8-4a21-b36c-105b6ef2bcc5 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:39:59.654613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a0fd6-b2e8-4a21-b36c-105b6ef2bcc5 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:39:59.655850 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 495/992] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:39:59.661174 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc74b86-c4ce-41a9-86a7-26075774854d None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:39:59.661603 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc74b86-c4ce-41a9-86a7-26075774854d None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:39:59.661710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc74b86-c4ce-41a9-86a7-26075774854d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:39:59.727308 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-cfc74b86-c4ce-41a9-86a7-26075774854d None None] MFA Rules not processed for user `9e8fef123072495eabcb616db3e670a6`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:39:59.730922 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-cfc74b86-c4ce-41a9-86a7-26075774854d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:39:59.782348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:39:59.828009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 498/993] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:39:59 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:40:00.114297 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a0fd6-b2e8-4a21-b36c-105b6ef2bcc5 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:40:00.115813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-058a0fd6-b2e8-4a21-b36c-105b6ef2bcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:00.126176 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:00.218805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a0fd6-b2e8-4a21-b36c-105b6ef2bcc5 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:40:00.291232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c39da8-f4de-4bcf-b0a4-1cfbcf273e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:40:00.291900 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c39da8-f4de-4bcf-b0a4-1cfbcf273e8f service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:00.292668 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c39da8-f4de-4bcf-b0a4-1cfbcf273e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:00.292940 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c39da8-f4de-4bcf-b0a4-1cfbcf273e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:00.294514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-58c39da8-f4de-4bcf-b0a4-1cfbcf273e8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:00.358566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:00.405778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c39da8-f4de-4bcf-b0a4-1cfbcf273e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:40:00.407220 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c39da8-f4de-4bcf-b0a4-1cfbcf273e8f service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:40:00.426098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 496/994] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:40:00 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:40:03.098825 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c523b-4b16-4e94-a40f-2002798f0e06 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:03.099395 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c523b-4b16-4e94-a40f-2002798f0e06 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:03.099395 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c523b-4b16-4e94-a40f-2002798f0e06 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:03.100749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 499/995] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:03.107024 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4c67a-0414-4e39-bff4-8375b58b5e42 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:03.107453 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4c67a-0414-4e39-bff4-8375b58b5e42 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:03.107578 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4c67a-0414-4e39-bff4-8375b58b5e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:03.173606 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-12f4c67a-0414-4e39-bff4-8375b58b5e42 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:03.174825 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-12f4c67a-0414-4e39-bff4-8375b58b5e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:03.269067 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 497/996] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:40:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:40:03.555603 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c523b-4b16-4e94-a40f-2002798f0e06 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:40:03.557170 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-9d2c523b-4b16-4e94-a40f-2002798f0e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:03.658634 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c523b-4b16-4e94-a40f-2002798f0e06 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:40:03.732462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b994a-8312-496d-a5cd-892a1c7dcb9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:40:03.733060 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b994a-8312-496d-a5cd-892a1c7dcb9d service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:03.733336 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b994a-8312-496d-a5cd-892a1c7dcb9d service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:03.733607 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b994a-8312-496d-a5cd-892a1c7dcb9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:03.735170 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-f56b994a-8312-496d-a5cd-892a1c7dcb9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:03.741993 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:03.845755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b994a-8312-496d-a5cd-892a1c7dcb9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:40:03.847107 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b994a-8312-496d-a5cd-892a1c7dcb9d service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:40:03.865941 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 500/997] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:40:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:40:06.688851 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73a5c8-d142-4941-a1de-ac56699aecab None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:06.689364 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73a5c8-d142-4941-a1de-ac56699aecab None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:06.689364 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73a5c8-d142-4941-a1de-ac56699aecab None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:06.690658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 498/998] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:06.696017 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54911f56-55e3-4514-aa05-4daaf30ec6d8 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:06.696528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54911f56-55e3-4514-aa05-4daaf30ec6d8 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:06.696614 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54911f56-55e3-4514-aa05-4daaf30ec6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:06.771209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-54911f56-55e3-4514-aa05-4daaf30ec6d8 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:06.772478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-54911f56-55e3-4514-aa05-4daaf30ec6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:06.836779 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:06.882428 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 501/999] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:40:07.161915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73a5c8-d142-4941-a1de-ac56699aecab None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:40:07.163459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ef73a5c8-d142-4941-a1de-ac56699aecab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:07.185246 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:07.270280 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73a5c8-d142-4941-a1de-ac56699aecab None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:40:07.344496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9186ff-dae2-40fe-8fa6-145f2d1477de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:40:07.345159 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9186ff-dae2-40fe-8fa6-145f2d1477de service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:07.345439 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9186ff-dae2-40fe-8fa6-145f2d1477de service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:07.345711 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9186ff-dae2-40fe-8fa6-145f2d1477de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:07.347278 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-1e9186ff-dae2-40fe-8fa6-145f2d1477de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:07.421327 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:07.459614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9186ff-dae2-40fe-8fa6-145f2d1477de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:40:07.461109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9186ff-dae2-40fe-8fa6-145f2d1477de service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:40:07.480300 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 499/1000] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:40:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:40:09.720484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160d85b-e38a-46c5-aa7e-9f6eb038427f None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:09.721014 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160d85b-e38a-46c5-aa7e-9f6eb038427f None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:09.721014 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160d85b-e38a-46c5-aa7e-9f6eb038427f None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:09.722384 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 502/1001] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:09.727657 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743d9dc-8d21-454e-b7e2-6b88ecc64cfd None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:09.728064 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743d9dc-8d21-454e-b7e2-6b88ecc64cfd None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:09.728259 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743d9dc-8d21-454e-b7e2-6b88ecc64cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:09.797218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-b743d9dc-8d21-454e-b7e2-6b88ecc64cfd None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:09.798779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-b743d9dc-8d21-454e-b7e2-6b88ecc64cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:09.907117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 500/1002] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:40:10.193543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160d85b-e38a-46c5-aa7e-9f6eb038427f None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:40:10.195054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-6160d85b-e38a-46c5-aa7e-9f6eb038427f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:10.297929 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160d85b-e38a-46c5-aa7e-9f6eb038427f None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:40:10.373022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e1364-b411-42b0-9b7f-005d1ab9a4bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:40:10.373631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e1364-b411-42b0-9b7f-005d1ab9a4bc service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:10.373909 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e1364-b411-42b0-9b7f-005d1ab9a4bc service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:10.374176 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e1364-b411-42b0-9b7f-005d1ab9a4bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:10.375762 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-500e1364-b411-42b0-9b7f-005d1ab9a4bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:10.393704 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:10.490385 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e1364-b411-42b0-9b7f-005d1ab9a4bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:40:10.491736 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e1364-b411-42b0-9b7f-005d1ab9a4bc service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:40:10.510197 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 503/1003] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:40:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:40:12.737079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8d4d4-b9b4-4db1-8cca-305b942991db None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:12.737621 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8d4d4-b9b4-4db1-8cca-305b942991db None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:12.737621 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8d4d4-b9b4-4db1-8cca-305b942991db None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:12.738970 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 501/1004] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:12.744362 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21c237-5c26-4bf3-abed-e7f668cb815d None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:12.744760 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21c237-5c26-4bf3-abed-e7f668cb815d None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:12.744868 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21c237-5c26-4bf3-abed-e7f668cb815d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:12.811511 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-eb21c237-5c26-4bf3-abed-e7f668cb815d None None] MFA Rules not processed for user `9e8fef123072495eabcb616db3e670a6`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:12.812747 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-eb21c237-5c26-4bf3-abed-e7f668cb815d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:12.879200 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:12.911761 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 504/1005] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:40:12 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:40:13.204183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8d4d4-b9b4-4db1-8cca-305b942991db None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:40:13.205715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f9f8d4d4-b9b4-4db1-8cca-305b942991db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:13.241737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:13.322196 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8d4d4-b9b4-4db1-8cca-305b942991db None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:40:13.397849 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ff09f-4104-45c8-aec4-6c8c44eaf875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:40:13.398468 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ff09f-4104-45c8-aec4-6c8c44eaf875 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:13.398748 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ff09f-4104-45c8-aec4-6c8c44eaf875 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:13.399023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ff09f-4104-45c8-aec4-6c8c44eaf875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:13.400640 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f38ff09f-4104-45c8-aec4-6c8c44eaf875 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:13.480792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:13.513323 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ff09f-4104-45c8-aec4-6c8c44eaf875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:40:13.514759 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ff09f-4104-45c8-aec4-6c8c44eaf875 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:40:13.533231 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 502/1006] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:40:13 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:40:15.916392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee3ab0-8e33-4144-aa13-29df7671ea51 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:15.916895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee3ab0-8e33-4144-aa13-29df7671ea51 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:15.917004 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee3ab0-8e33-4144-aa13-29df7671ea51 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:15.918941 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 505/1007] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:40:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:15.924396 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177bc46-327e-443b-8a2b-b07e4d55017a None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:15.924792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177bc46-327e-443b-8a2b-b07e4d55017a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:15.924944 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177bc46-327e-443b-8a2b-b07e4d55017a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:15.990741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-f177bc46-327e-443b-8a2b-b07e4d55017a None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:15.991936 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f177bc46-327e-443b-8a2b-b07e4d55017a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:16.086608 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 503/1008] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:40:16.371872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee3ab0-8e33-4144-aa13-29df7671ea51 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:40:16.373436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-2fee3ab0-8e33-4144-aa13-29df7671ea51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:16.476412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee3ab0-8e33-4144-aa13-29df7671ea51 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:40:16.549581 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed012950-2d47-49d0-b459-63d68fa4e6c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:40:16.550170 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed012950-2d47-49d0-b459-63d68fa4e6c8 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:16.550444 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed012950-2d47-49d0-b459-63d68fa4e6c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:16.550710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed012950-2d47-49d0-b459-63d68fa4e6c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:16.552293 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-ed012950-2d47-49d0-b459-63d68fa4e6c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:16.579096 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:16.660248 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed012950-2d47-49d0-b459-63d68fa4e6c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:40:16.661692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed012950-2d47-49d0-b459-63d68fa4e6c8 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:40:16.680693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 506/1009] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:40:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:40:18.870029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc9b7b-f412-4c7a-b7ac-7dbc8d7e7ab2 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:18.870646 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc9b7b-f412-4c7a-b7ac-7dbc8d7e7ab2 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:18.870646 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc9b7b-f412-4c7a-b7ac-7dbc8d7e7ab2 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:18.871891 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 504/1010] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:18.877292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0776e8d-8a76-4ff5-a0b6-3a78c5c63457 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:18.877738 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0776e8d-8a76-4ff5-a0b6-3a78c5c63457 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:18.877830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0776e8d-8a76-4ff5-a0b6-3a78c5c63457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:18.943273 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-e0776e8d-8a76-4ff5-a0b6-3a78c5c63457 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:18.944516 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e0776e8d-8a76-4ff5-a0b6-3a78c5c63457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:19.017840 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:19.039902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 507/1011] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:40:19.340249 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1cf71-3c80-4100-8508-c40c5a79ca78 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:19.340777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1cf71-3c80-4100-8508-c40c5a79ca78 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:19.341159 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1cf71-3c80-4100-8508-c40c5a79ca78 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:19.342518 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 505/1012] 10.222.0.36 () {54 vars in 965 bytes} [Sun Oct 30 00:40:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:19.348307 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f7ea5-339d-450c-8a54-5a6a4bbd8cd8 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:19.348597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f7ea5-339d-450c-8a54-5a6a4bbd8cd8 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:19.348879 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f7ea5-339d-450c-8a54-5a6a4bbd8cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:19.426751 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-1b3f7ea5-339d-450c-8a54-5a6a4bbd8cd8 None None] MFA Rules not processed for user `fb39b6cc97a1456bb1312f49118c6e10`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:19.427945 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-1b3f7ea5-339d-450c-8a54-5a6a4bbd8cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:19.504308 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 508/1013] 10.222.0.36 () {58 vars in 1062 bytes} [Sun Oct 30 00:40:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:40:19.514228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1cf71-3c80-4100-8508-c40c5a79ca78 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:40:19.515740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-aca1cf71-3c80-4100-8508-c40c5a79ca78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:19.555399 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:19.619134 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1cf71-3c80-4100-8508-c40c5a79ca78 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:40:19.692909 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f1f50-2e35-4521-a6c6-9f9dd830b2d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:40:19.693514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f1f50-2e35-4521-a6c6-9f9dd830b2d7 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:19.693792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f1f50-2e35-4521-a6c6-9f9dd830b2d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:19.694062 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f1f50-2e35-4521-a6c6-9f9dd830b2d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:19.695630 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-030f1f50-2e35-4521-a6c6-9f9dd830b2d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:19.779608 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:19.805755 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f1f50-2e35-4521-a6c6-9f9dd830b2d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:40:19.807103 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f1f50-2e35-4521-a6c6-9f9dd830b2d7 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:40:19.825191 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 506/1014] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:40:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:40:22.818228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa61450-9c55-45a9-91df-e702ec72f7b0 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:22.818755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa61450-9c55-45a9-91df-e702ec72f7b0 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:22.818755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa61450-9c55-45a9-91df-e702ec72f7b0 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:22.820033 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 509/1015] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:22.825449 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9812338-c203-45dc-ae77-1fe1c9fddd17 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:22.825867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9812338-c203-45dc-ae77-1fe1c9fddd17 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:22.825978 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9812338-c203-45dc-ae77-1fe1c9fddd17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:22.893105 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-e9812338-c203-45dc-ae77-1fe1c9fddd17 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:22.894333 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e9812338-c203-45dc-ae77-1fe1c9fddd17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:22.990426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 507/1016] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:40:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:40:23.278806 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa61450-9c55-45a9-91df-e702ec72f7b0 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:40:23.280370 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-dfa61450-9c55-45a9-91df-e702ec72f7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:23.357681 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:23.385297 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa61450-9c55-45a9-91df-e702ec72f7b0 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:40:23.461048 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2157488-d3a5-4cf1-845f-e0233cfb76b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:40:23.461688 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2157488-d3a5-4cf1-845f-e0233cfb76b3 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:23.461970 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2157488-d3a5-4cf1-845f-e0233cfb76b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:23.462246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2157488-d3a5-4cf1-845f-e0233cfb76b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:23.463835 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-f2157488-d3a5-4cf1-845f-e0233cfb76b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:23.572580 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2157488-d3a5-4cf1-845f-e0233cfb76b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:40:23.574081 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2157488-d3a5-4cf1-845f-e0233cfb76b3 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:40:23.592239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 510/1017] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:40:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:40:27.812741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd1d5b-c743-44ce-8c4d-4d5d2b6f66aa None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:27.813238 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd1d5b-c743-44ce-8c4d-4d5d2b6f66aa None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:27.813238 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd1d5b-c743-44ce-8c4d-4d5d2b6f66aa None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:27.814564 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 508/1018] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:27.819923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631eb64-b1a4-4593-aee2-f25ebd1a0317 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:27.820385 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631eb64-b1a4-4593-aee2-f25ebd1a0317 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:27.820715 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631eb64-b1a4-4593-aee2-f25ebd1a0317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:27.826383 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:27.890762 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-0631eb64-b1a4-4593-aee2-f25ebd1a0317 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:27.892037 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-0631eb64-b1a4-4593-aee2-f25ebd1a0317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:27.990582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 511/1019] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:40:28.279602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd1d5b-c743-44ce-8c4d-4d5d2b6f66aa None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:40:28.281170 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-36bd1d5b-c743-44ce-8c4d-4d5d2b6f66aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:28.330036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:28.385086 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd1d5b-c743-44ce-8c4d-4d5d2b6f66aa None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:40:28.464017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d86b2-a13c-44ea-afec-744c96fe81be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:40:28.464783 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d86b2-a13c-44ea-afec-744c96fe81be service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:28.465052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d86b2-a13c-44ea-afec-744c96fe81be service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:28.465347 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d86b2-a13c-44ea-afec-744c96fe81be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:28.466953 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-5f8d86b2-a13c-44ea-afec-744c96fe81be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:28.561393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:28.578523 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d86b2-a13c-44ea-afec-744c96fe81be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:40:28.579979 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d86b2-a13c-44ea-afec-744c96fe81be service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:40:28.599104 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 509/1020] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:40:31.401214 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a210aa-435c-450d-a3cf-1a33609b9965 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:31.401787 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a210aa-435c-450d-a3cf-1a33609b9965 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:31.401787 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a210aa-435c-450d-a3cf-1a33609b9965 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:31.403464 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 512/1021] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:31.409372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6eae61-31ea-4835-a433-fd58355b0b20 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:31.409836 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6eae61-31ea-4835-a433-fd58355b0b20 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:31.410049 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6eae61-31ea-4835-a433-fd58355b0b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:31.484814 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-7c6eae61-31ea-4835-a433-fd58355b0b20 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:31.486005 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-7c6eae61-31ea-4835-a433-fd58355b0b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:31.585445 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 510/1022] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:40:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:40:31.873090 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a210aa-435c-450d-a3cf-1a33609b9965 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:40:31.874698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-46a210aa-435c-450d-a3cf-1a33609b9965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:31.969455 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:31.989059 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a210aa-435c-450d-a3cf-1a33609b9965 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:40:32.064977 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303bf46-f0bb-43ba-a6a4-73c07e6dc72c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:40:32.065580 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303bf46-f0bb-43ba-a6a4-73c07e6dc72c service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:32.065857 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303bf46-f0bb-43ba-a6a4-73c07e6dc72c service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:32.066127 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303bf46-f0bb-43ba-a6a4-73c07e6dc72c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:32.067707 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-8303bf46-f0bb-43ba-a6a4-73c07e6dc72c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:32.176498 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303bf46-f0bb-43ba-a6a4-73c07e6dc72c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:40:32.177944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303bf46-f0bb-43ba-a6a4-73c07e6dc72c service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:40:32.196606 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 513/1023] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:40:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:40:36.336968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6dc48-a4bc-4d0a-8e2a-57dfa2943815 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:36.337514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6dc48-a4bc-4d0a-8e2a-57dfa2943815 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:36.337610 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6dc48-a4bc-4d0a-8e2a-57dfa2943815 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:36.338958 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 511/1024] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:36.344780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970d927-650a-4699-8237-fd6fa3b15c6c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:36.345217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970d927-650a-4699-8237-fd6fa3b15c6c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:36.345363 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970d927-650a-4699-8237-fd6fa3b15c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:36.361144 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:36.419644 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-c970d927-650a-4699-8237-fd6fa3b15c6c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:36.420537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c970d927-650a-4699-8237-fd6fa3b15c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:36.528985 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 514/1025] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:40:36.815194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6dc48-a4bc-4d0a-8e2a-57dfa2943815 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:40:36.816745 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-eaa6dc48-a4bc-4d0a-8e2a-57dfa2943815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:36.874700 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:36.921775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6dc48-a4bc-4d0a-8e2a-57dfa2943815 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:40:36.995579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb090d0-c87a-47b4-9556-765a1e2a0fc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:40:36.996722 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb090d0-c87a-47b4-9556-765a1e2a0fc1 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:36.997597 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb090d0-c87a-47b4-9556-765a1e2a0fc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:36.997873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb090d0-c87a-47b4-9556-765a1e2a0fc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:36.999455 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-bbb090d0-c87a-47b4-9556-765a1e2a0fc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:37.105840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:37.110320 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb090d0-c87a-47b4-9556-765a1e2a0fc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:40:37.111762 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb090d0-c87a-47b4-9556-765a1e2a0fc1 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:40:37.130314 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 512/1026] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:40:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:40:42.153487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3d242-4393-43a7-8842-f94349a6f6d0 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:42.154379 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3d242-4393-43a7-8842-f94349a6f6d0 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:42.154823 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3d242-4393-43a7-8842-f94349a6f6d0 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:42.156380 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 515/1027] 10.222.0.36 () {54 vars in 921 bytes} [Sun Oct 30 00:40:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:42.165856 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e9e6e-6729-4f5e-9e75-cdefa7f6138c None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:42.166297 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e9e6e-6729-4f5e-9e75-cdefa7f6138c None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:42.166397 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e9e6e-6729-4f5e-9e75-cdefa7f6138c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:42.295343 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-254e9e6e-6729-4f5e-9e75-cdefa7f6138c None None] MFA Rules not processed for user `5ded89d48d6c4a15a5c5829f594f8776`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:42.297405 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-254e9e6e-6729-4f5e-9e75-cdefa7f6138c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:42.425915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 513/1028] 10.222.0.36 () {58 vars in 1018 bytes} [Sun Oct 30 00:40:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:40:46.527546 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=59 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:46.624299 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9ad18-a93b-45c9-b753-206f516ead21 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:46.624900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9ad18-a93b-45c9-b753-206f516ead21 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:46.625295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9ad18-a93b-45c9-b753-206f516ead21 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:46.627009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 516/1029] 10.222.0.36 () {54 vars in 921 bytes} [Sun Oct 30 00:40:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:46.632868 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdbc96-0098-410c-a1e6-f4877617bc5d None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:46.633413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdbc96-0098-410c-a1e6-f4877617bc5d None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:46.633712 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdbc96-0098-410c-a1e6-f4877617bc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:46.702856 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:46.732303 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-a1bdbc96-0098-410c-a1e6-f4877617bc5d None None] MFA Rules not processed for user `5ded89d48d6c4a15a5c5829f594f8776`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:46.733805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a1bdbc96-0098-410c-a1e6-f4877617bc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:46.867008 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 514/1030] 10.222.0.36 () {58 vars in 1018 bytes} [Sun Oct 30 00:40:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:40:48.335593 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df60342-1943-4a76-8a5c-fdc371367280 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:48.336310 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df60342-1943-4a76-8a5c-fdc371367280 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:48.336441 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df60342-1943-4a76-8a5c-fdc371367280 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:48.337683 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 517/1031] 10.222.0.36 () {54 vars in 921 bytes} [Sun Oct 30 00:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:48.343544 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b2265-44cb-4878-9d3d-f071c63d64b8 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:48.343988 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b2265-44cb-4878-9d3d-f071c63d64b8 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:48.344282 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b2265-44cb-4878-9d3d-f071c63d64b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:48.424714 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-0c0b2265-44cb-4878-9d3d-f071c63d64b8 None None] MFA Rules not processed for user `5ded89d48d6c4a15a5c5829f594f8776`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:48.425923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-0c0b2265-44cb-4878-9d3d-f071c63d64b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:48.471103 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:48.536289 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 515/1032] 10.222.0.36 () {58 vars in 1018 bytes} [Sun Oct 30 00:40:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:40:52.041453 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147bd79-0aa2-4ef1-bc81-a50a10977362 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:52.041947 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147bd79-0aa2-4ef1-bc81-a50a10977362 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:52.041947 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147bd79-0aa2-4ef1-bc81-a50a10977362 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:52.043429 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 518/1033] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:52.050509 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d496a0-92ba-4321-bb17-e28f21b1a3a9 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:52.051013 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d496a0-92ba-4321-bb17-e28f21b1a3a9 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:52.051092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d496a0-92ba-4321-bb17-e28f21b1a3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:52.133686 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-74d496a0-92ba-4321-bb17-e28f21b1a3a9 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:52.134905 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-74d496a0-92ba-4321-bb17-e28f21b1a3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:52.249138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 516/1034] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:40:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:40:52.562611 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f46ac-0dc2-49d8-84ff-c25d58ff55b7 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:52.563084 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f46ac-0dc2-49d8-84ff-c25d58ff55b7 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:52.563516 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f46ac-0dc2-49d8-84ff-c25d58ff55b7 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:52.565618 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 519/1035] 10.222.0.36 () {54 vars in 961 bytes} [Sun Oct 30 00:40:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:52.572353 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4b85c-ae88-4be8-8a3c-f7c1d592722f None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:52.572842 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4b85c-ae88-4be8-8a3c-f7c1d592722f None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:52.573278 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4b85c-ae88-4be8-8a3c-f7c1d592722f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:52.592533 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:52.666415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-59f4b85c-ae88-4be8-8a3c-f7c1d592722f None None] MFA Rules not processed for user `adbba7be39a64230bab160ce77ce7586`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:52.667781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-59f4b85c-ae88-4be8-8a3c-f7c1d592722f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:52.796540 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 517/1036] 10.222.0.36 () {58 vars in 1058 bytes} [Sun Oct 30 00:40:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:40:52.805076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f46ac-0dc2-49d8-84ff-c25d58ff55b7 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:40:52.806612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-715f46ac-0dc2-49d8-84ff-c25d58ff55b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:52.965077 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f46ac-0dc2-49d8-84ff-c25d58ff55b7 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:40:53.042898 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:53.087173 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016217e9-2f13-4aa5-9f89-2c052aea844d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:40:53.087838 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016217e9-2f13-4aa5-9f89-2c052aea844d service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:53.088253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016217e9-2f13-4aa5-9f89-2c052aea844d service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:53.088621 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016217e9-2f13-4aa5-9f89-2c052aea844d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:53.090285 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-016217e9-2f13-4aa5-9f89-2c052aea844d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:53.190570 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016217e9-2f13-4aa5-9f89-2c052aea844d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:40:53.192340 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016217e9-2f13-4aa5-9f89-2c052aea844d service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:40:53.211777 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 520/1037] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:40:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:40:55.645644 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88933ab6-2bd2-4e08-90db-1146185c7c52 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:55.646157 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88933ab6-2bd2-4e08-90db-1146185c7c52 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:55.646835 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88933ab6-2bd2-4e08-90db-1146185c7c52 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:55.657207 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 518/1038] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:40:55 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:55.662048 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78920b9-98da-4cb2-9811-4275bf11f778 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:55.662494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78920b9-98da-4cb2-9811-4275bf11f778 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:55.662494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78920b9-98da-4cb2-9811-4275bf11f778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:55.741249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-e78920b9-98da-4cb2-9811-4275bf11f778 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:55.742512 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e78920b9-98da-4cb2-9811-4275bf11f778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:55.785664 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:55.854858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 521/1039] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:40:56.147536 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88933ab6-2bd2-4e08-90db-1146185c7c52 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:40:56.149444 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-88933ab6-2bd2-4e08-90db-1146185c7c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:56.220149 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:56.324460 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88933ab6-2bd2-4e08-90db-1146185c7c52 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:40:56.454195 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b1974-4d42-45ad-b9ce-d76ce281768f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:40:56.454818 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b1974-4d42-45ad-b9ce-d76ce281768f service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:56.455095 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b1974-4d42-45ad-b9ce-d76ce281768f service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:56.455365 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b1974-4d42-45ad-b9ce-d76ce281768f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:56.457008 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-6e3b1974-4d42-45ad-b9ce-d76ce281768f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:56.480990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:56.550856 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b1974-4d42-45ad-b9ce-d76ce281768f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:40:56.552748 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b1974-4d42-45ad-b9ce-d76ce281768f service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:40:56.572248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 519/1040] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:40:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:40:59.066743 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa7894-4586-4c93-a21f-ca8dfb56e028 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:59.067256 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa7894-4586-4c93-a21f-ca8dfb56e028 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:59.067256 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa7894-4586-4c93-a21f-ca8dfb56e028 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:59.068605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 522/1041] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:59.073964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edcfe5-514b-45f3-94e0-2bd41012888b None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:59.074374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edcfe5-514b-45f3-94e0-2bd41012888b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:59.074543 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edcfe5-514b-45f3-94e0-2bd41012888b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:59.143463 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-e7edcfe5-514b-45f3-94e0-2bd41012888b None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:59.144696 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e7edcfe5-514b-45f3-94e0-2bd41012888b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:59.242541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:40:59.244989 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 520/1042] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:40:59.542180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f9738-4fe6-404a-a2df-aa5146acd9bc None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:59.542769 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f9738-4fe6-404a-a2df-aa5146acd9bc None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:59.543154 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f9738-4fe6-404a-a2df-aa5146acd9bc None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:59.545256 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 523/1043] 10.222.0.36 () {54 vars in 961 bytes} [Sun Oct 30 00:40:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:40:59.551372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f7792-7c49-4ec5-8084-0734517e1df3 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:40:59.551742 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f7792-7c49-4ec5-8084-0734517e1df3 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:40:59.552363 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f7792-7c49-4ec5-8084-0734517e1df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:40:59.633925 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-a03f7792-7c49-4ec5-8084-0734517e1df3 None None] MFA Rules not processed for user `adbba7be39a64230bab160ce77ce7586`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:40:59.635391 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a03f7792-7c49-4ec5-8084-0734517e1df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:59.759476 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 521/1044] 10.222.0.36 () {58 vars in 1058 bytes} [Sun Oct 30 00:40:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:40:59.767950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f9738-4fe6-404a-a2df-aa5146acd9bc None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:40:59.769646 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4a7f9738-4fe6-404a-a2df-aa5146acd9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:40:59.953076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f9738-4fe6-404a-a2df-aa5146acd9bc None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:40:59.975518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:00.077633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cdc0a2-176d-4e36-86c4-4d09c1a576f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:00.078414 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdc0a2-176d-4e36-86c4-4d09c1a576f1 service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:00.078502 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdc0a2-176d-4e36-86c4-4d09c1a576f1 service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:00.078758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdc0a2-176d-4e36-86c4-4d09c1a576f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:00.080365 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b1cdc0a2-176d-4e36-86c4-4d09c1a576f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:00.178155 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cdc0a2-176d-4e36-86c4-4d09c1a576f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:00.179780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cdc0a2-176d-4e36-86c4-4d09c1a576f1 service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:00.205217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 524/1045] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:40:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:41:00.316690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cae14-1da9-42dd-992a-bf22c3dd48b9 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:00.317270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cae14-1da9-42dd-992a-bf22c3dd48b9 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:00.317270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cae14-1da9-42dd-992a-bf22c3dd48b9 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:00.318971 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 522/1046] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:00.324701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61a837-7121-4ec2-8b61-c43707fc622a None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:00.325089 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61a837-7121-4ec2-8b61-c43707fc622a None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:00.325890 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61a837-7121-4ec2-8b61-c43707fc622a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:00.361735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:00.410363 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-9a61a837-7121-4ec2-8b61-c43707fc622a None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:00.411685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-9a61a837-7121-4ec2-8b61-c43707fc622a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:00.548863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 525/1047] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:41:00.911028 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48318faa-b552-4415-8fcb-9eec88ee4a86 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:00.911319 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48318faa-b552-4415-8fcb-9eec88ee4a86 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:00.911596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48318faa-b552-4415-8fcb-9eec88ee4a86 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:00.913169 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 523/1048] 10.222.0.36 () {54 vars in 963 bytes} [Sun Oct 30 00:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:00.917856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5a884-8002-4d4b-8034-d26f2b4ccba9 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:00.918387 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5a884-8002-4d4b-8034-d26f2b4ccba9 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:00.918939 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5a884-8002-4d4b-8034-d26f2b4ccba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:01.018983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-8bd5a884-8002-4d4b-8034-d26f2b4ccba9 None None] MFA Rules not processed for user `2dbe6b3bdfb3405589b8ce5aebebb93e`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:01.020229 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-8bd5a884-8002-4d4b-8034-d26f2b4ccba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:01.070649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:01.105186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 526/1049] 10.222.0.36 () {58 vars in 1060 bytes} [Sun Oct 30 00:41:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:41:01.114158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48318faa-b552-4415-8fcb-9eec88ee4a86 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:01.115952 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-48318faa-b552-4415-8fcb-9eec88ee4a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:01.172591 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:01.230521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48318faa-b552-4415-8fcb-9eec88ee4a86 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:01.313966 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55c1e1-4001-40b6-a2c6-d3fb51565d14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:01.314618 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55c1e1-4001-40b6-a2c6-d3fb51565d14 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:01.314864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55c1e1-4001-40b6-a2c6-d3fb51565d14 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:01.315141 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55c1e1-4001-40b6-a2c6-d3fb51565d14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:01.317048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-be55c1e1-4001-40b6-a2c6-d3fb51565d14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:01.439498 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:01.459718 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55c1e1-4001-40b6-a2c6-d3fb51565d14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:01.463744 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55c1e1-4001-40b6-a2c6-d3fb51565d14 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:01.507838 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 524/1050] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:41:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:41:02.497915 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca792c-9f3b-4f19-b95d-aed95cf27186 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:02.498946 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca792c-9f3b-4f19-b95d-aed95cf27186 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:02.498946 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca792c-9f3b-4f19-b95d-aed95cf27186 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:02.500064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 527/1051] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:41:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:02.505244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d5b0d-8fc6-48cf-b2a0-ca225e2e93a6 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:02.505656 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d5b0d-8fc6-48cf-b2a0-ca225e2e93a6 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:02.505833 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d5b0d-8fc6-48cf-b2a0-ca225e2e93a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:02.574482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-5a8d5b0d-8fc6-48cf-b2a0-ca225e2e93a6 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:02.576958 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-5a8d5b0d-8fc6-48cf-b2a0-ca225e2e93a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:02.674774 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 525/1052] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:41:02.965768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca792c-9f3b-4f19-b95d-aed95cf27186 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:02.978678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-61ca792c-9f3b-4f19-b95d-aed95cf27186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:03.140326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca792c-9f3b-4f19-b95d-aed95cf27186 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:03.195946 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:03.261008 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce313e79-b2a3-4457-8f12-280edfcded28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:03.261622 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce313e79-b2a3-4457-8f12-280edfcded28 service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:03.261902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce313e79-b2a3-4457-8f12-280edfcded28 service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:03.262173 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce313e79-b2a3-4457-8f12-280edfcded28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:03.264395 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-ce313e79-b2a3-4457-8f12-280edfcded28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:03.357967 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce313e79-b2a3-4457-8f12-280edfcded28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:03.359310 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce313e79-b2a3-4457-8f12-280edfcded28 service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:03.378316 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 528/1053] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:41:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:41:05.640244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca95c7-3eaf-4fa1-916d-24c7b0f101b5 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:05.640830 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca95c7-3eaf-4fa1-916d-24c7b0f101b5 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:05.640830 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca95c7-3eaf-4fa1-916d-24c7b0f101b5 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:05.642320 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 526/1054] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:05.647463 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c15cdb-83a1-4109-90cc-a755808d019f None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:05.647884 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c15cdb-83a1-4109-90cc-a755808d019f None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:05.647999 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c15cdb-83a1-4109-90cc-a755808d019f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:05.716655 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-f5c15cdb-83a1-4109-90cc-a755808d019f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:05.717856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-f5c15cdb-83a1-4109-90cc-a755808d019f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:05.725460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:05.834536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 529/1055] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:41:06.125733 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca95c7-3eaf-4fa1-916d-24c7b0f101b5 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:06.127362 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-55ca95c7-3eaf-4fa1-916d-24c7b0f101b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:06.194294 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:06.288115 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca95c7-3eaf-4fa1-916d-24c7b0f101b5 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:06.417670 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469228e-102e-464d-978e-27282876043b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:06.420073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469228e-102e-464d-978e-27282876043b service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:06.420632 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469228e-102e-464d-978e-27282876043b service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:06.421238 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469228e-102e-464d-978e-27282876043b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:06.423611 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-b469228e-102e-464d-978e-27282876043b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:06.457793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:06.545711 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469228e-102e-464d-978e-27282876043b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:06.547411 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469228e-102e-464d-978e-27282876043b service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:06.566948 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 527/1056] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:41:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:41:08.823848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8334ee2-4c1e-4f87-b44c-2b795257f40b None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:08.824330 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8334ee2-4c1e-4f87-b44c-2b795257f40b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:08.824460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8334ee2-4c1e-4f87-b44c-2b795257f40b None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:08.825830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 530/1057] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:08.831242 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f930b-6f2d-4fbb-9f3d-ba5a8d30bdfc None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:08.831673 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f930b-6f2d-4fbb-9f3d-ba5a8d30bdfc None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:08.831769 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f930b-6f2d-4fbb-9f3d-ba5a8d30bdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:08.907023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-de7f930b-6f2d-4fbb-9f3d-ba5a8d30bdfc None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:08.908236 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-de7f930b-6f2d-4fbb-9f3d-ba5a8d30bdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:09.010199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:09.018519 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 528/1058] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:41:09.333427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8334ee2-4c1e-4f87-b44c-2b795257f40b None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:09.334930 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-f8334ee2-4c1e-4f87-b44c-2b795257f40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:09.493612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:09.508879 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8334ee2-4c1e-4f87-b44c-2b795257f40b None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:09.656144 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34f5f3-297f-4ee1-98e3-3afb3c1e3915 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:09.657414 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34f5f3-297f-4ee1-98e3-3afb3c1e3915 service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:09.658077 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34f5f3-297f-4ee1-98e3-3afb3c1e3915 service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:09.658752 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34f5f3-297f-4ee1-98e3-3afb3c1e3915 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:09.660834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-be34f5f3-297f-4ee1-98e3-3afb3c1e3915 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:09.775078 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34f5f3-297f-4ee1-98e3-3afb3c1e3915 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:09.777442 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34f5f3-297f-4ee1-98e3-3afb3c1e3915 service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:09.793450 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:09.807495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 531/1059] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:41:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:41:12.083655 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228612a-6c06-417a-839d-74847655569a None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:12.084329 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228612a-6c06-417a-839d-74847655569a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:12.084329 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228612a-6c06-417a-839d-74847655569a None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:12.085554 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 529/1060] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:12.091190 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2680e-b9fc-414d-8dfd-b8f2eef8b991 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:12.092128 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2680e-b9fc-414d-8dfd-b8f2eef8b991 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:12.092783 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2680e-b9fc-414d-8dfd-b8f2eef8b991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:12.164562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-e3e2680e-b9fc-414d-8dfd-b8f2eef8b991 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:12.165807 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e3e2680e-b9fc-414d-8dfd-b8f2eef8b991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:12.271040 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 532/1061] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:41:12.621076 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228612a-6c06-417a-839d-74847655569a None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:12.623503 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e228612a-6c06-417a-839d-74847655569a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:12.786080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228612a-6c06-417a-839d-74847655569a None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:12.853539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:12.900573 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33d3e3-6a48-4fde-a5b0-01d1dbbd8924 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:12.901186 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33d3e3-6a48-4fde-a5b0-01d1dbbd8924 service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:12.901466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33d3e3-6a48-4fde-a5b0-01d1dbbd8924 service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:12.901740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33d3e3-6a48-4fde-a5b0-01d1dbbd8924 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:12.903333 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-cf33d3e3-6a48-4fde-a5b0-01d1dbbd8924 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:13.005117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf33d3e3-6a48-4fde-a5b0-01d1dbbd8924 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:13.006917 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf33d3e3-6a48-4fde-a5b0-01d1dbbd8924 service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:13.025278 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 530/1062] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:41:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:41:15.264363 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c245f07-6472-495a-9a5f-e4a24e7daaaa None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:15.264900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c245f07-6472-495a-9a5f-e4a24e7daaaa None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:15.264900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c245f07-6472-495a-9a5f-e4a24e7daaaa None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:15.266238 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 533/1063] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:15.272352 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84205b-8089-4a34-806d-b7504ae8fff6 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:15.272856 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84205b-8089-4a34-806d-b7504ae8fff6 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:15.273055 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84205b-8089-4a34-806d-b7504ae8fff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:15.347017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-5f84205b-8089-4a34-806d-b7504ae8fff6 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:15.348532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-5f84205b-8089-4a34-806d-b7504ae8fff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:15.384815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:15.453205 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 531/1064] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:41:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:41:15.754283 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c245f07-6472-495a-9a5f-e4a24e7daaaa None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:15.756258 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4c245f07-6472-495a-9a5f-e4a24e7daaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:15.848956 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:15.928119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c245f07-6472-495a-9a5f-e4a24e7daaaa None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:16.051626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fa5f3-3d6a-4ee1-b241-f6c9dedb680c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:16.052382 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fa5f3-3d6a-4ee1-b241-f6c9dedb680c service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:16.052542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fa5f3-3d6a-4ee1-b241-f6c9dedb680c service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:16.052808 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fa5f3-3d6a-4ee1-b241-f6c9dedb680c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:16.054645 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b91fa5f3-3d6a-4ee1-b241-f6c9dedb680c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:16.095177 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:16.148648 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fa5f3-3d6a-4ee1-b241-f6c9dedb680c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:16.150252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fa5f3-3d6a-4ee1-b241-f6c9dedb680c service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:16.169550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 534/1065] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:41:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:41:18.367513 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a762bd-e6ef-416d-bb57-e096f45c4797 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:18.368127 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a762bd-e6ef-416d-bb57-e096f45c4797 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:18.368127 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a762bd-e6ef-416d-bb57-e096f45c4797 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:18.369488 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 532/1066] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:18.374990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f776f-52ee-4587-9f3b-c747f8ab5a02 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:18.375423 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f776f-52ee-4587-9f3b-c747f8ab5a02 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:18.375518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f776f-52ee-4587-9f3b-c747f8ab5a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:18.442452 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-758f776f-52ee-4587-9f3b-c747f8ab5a02 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:18.443665 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-758f776f-52ee-4587-9f3b-c747f8ab5a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:18.542771 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 535/1067] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:41:18.845237 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a762bd-e6ef-416d-bb57-e096f45c4797 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:18.846814 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-b3a762bd-e6ef-416d-bb57-e096f45c4797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:18.999007 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a762bd-e6ef-416d-bb57-e096f45c4797 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:19.016205 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:19.118531 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4aba2e-8845-4601-b9f8-44ac5bd0c33e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:19.119164 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4aba2e-8845-4601-b9f8-44ac5bd0c33e service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:19.119451 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4aba2e-8845-4601-b9f8-44ac5bd0c33e service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:19.119720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4aba2e-8845-4601-b9f8-44ac5bd0c33e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:19.121327 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-cc4aba2e-8845-4601-b9f8-44ac5bd0c33e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:19.166424 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26952f4-def1-46c8-8a42-1f01f2a593bf None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:19.166708 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26952f4-def1-46c8-8a42-1f01f2a593bf None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:19.166988 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26952f4-def1-46c8-8a42-1f01f2a593bf None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:19.168363 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 536/1068] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:41:19.174168 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879bc0e-3fbe-4458-8482-fac86dc273ef None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:19.174451 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879bc0e-3fbe-4458-8482-fac86dc273ef None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:19.174731 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879bc0e-3fbe-4458-8482-fac86dc273ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:19.191072 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:19.215693 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4aba2e-8845-4601-b9f8-44ac5bd0c33e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:19.217590 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4aba2e-8845-4601-b9f8-44ac5bd0c33e service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:19.236113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 533/1069] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:41:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:41:19.247904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-7879bc0e-3fbe-4458-8482-fac86dc273ef None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:19.249394 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-7879bc0e-3fbe-4458-8482-fac86dc273ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:19.363227 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 537/1070] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:41:19.673689 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b3195-8d19-44ba-8ee6-74db065f3614 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:19.674187 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b3195-8d19-44ba-8ee6-74db065f3614 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:19.674320 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b3195-8d19-44ba-8ee6-74db065f3614 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:19.675745 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 534/1071] 10.222.0.36 () {54 vars in 962 bytes} [Sun Oct 30 00:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:19.681350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05093d4-eac8-49fb-8ff8-4e92d559486c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:19.681771 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05093d4-eac8-49fb-8ff8-4e92d559486c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:19.681915 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05093d4-eac8-49fb-8ff8-4e92d559486c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:19.760745 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-c05093d4-eac8-49fb-8ff8-4e92d559486c None None] MFA Rules not processed for user `7cf9451f2ef7478589b4a4278cb03252`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:19.761958 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c05093d4-eac8-49fb-8ff8-4e92d559486c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:19.797774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:19.847741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 538/1072] 10.222.0.36 () {58 vars in 1059 bytes} [Sun Oct 30 00:41:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:41:19.855772 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b3195-8d19-44ba-8ee6-74db065f3614 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:19.857300 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c71b3195-8d19-44ba-8ee6-74db065f3614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:19.884010 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:19.969480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b3195-8d19-44ba-8ee6-74db065f3614 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:20.046447 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291a9ad-4680-4500-a06a-ba132c81a7f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cf9451f2ef7478589b4a4278cb03252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:20.047055 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291a9ad-4680-4500-a06a-ba132c81a7f5 service glance] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:20.047338 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291a9ad-4680-4500-a06a-ba132c81a7f5 service glance] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:20.047613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291a9ad-4680-4500-a06a-ba132c81a7f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:20.049260 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d291a9ad-4680-4500-a06a-ba132c81a7f5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:20.122480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:20.159175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291a9ad-4680-4500-a06a-ba132c81a7f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:20.160656 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291a9ad-4680-4500-a06a-ba132c81a7f5 service glance] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:20.180586 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 535/1073] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:41:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:41:20.930040 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401c5f8-728b-40c2-b829-071909378f92 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:20.930501 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401c5f8-728b-40c2-b829-071909378f92 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:20.930780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401c5f8-728b-40c2-b829-071909378f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:21.003121 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-6401c5f8-728b-40c2-b829-071909378f92 None None] MFA Rules not processed for user `8a91d3b9c4834d729211ec367e235889`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:21.004526 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-6401c5f8-728b-40c2-b829-071909378f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:21.094753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 539/1074] 10.222.0.36 () {58 vars in 1009 bytes} [Sun Oct 30 00:41:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:41:21.141789 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50200eef-d3ec-42c6-bfd9-6220f1bfe93b None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:21.142259 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50200eef-d3ec-42c6-bfd9-6220f1bfe93b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:21.142383 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50200eef-d3ec-42c6-bfd9-6220f1bfe93b None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:21.143662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 536/1075] 10.222.0.36 () {54 vars in 943 bytes} [Sun Oct 30 00:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:21.149880 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5308b2e-21c4-489b-8ea2-e38942a793ea None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:21.150175 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5308b2e-21c4-489b-8ea2-e38942a793ea None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:21.150454 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5308b2e-21c4-489b-8ea2-e38942a793ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:21.209569 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:21.229182 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-a5308b2e-21c4-489b-8ea2-e38942a793ea None None] MFA Rules not processed for user `9f4c779b8422432fb06cdf996841b9b5`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:21.230401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a5308b2e-21c4-489b-8ea2-e38942a793ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:21.339672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 540/1076] 10.222.0.36 () {58 vars in 1040 bytes} [Sun Oct 30 00:41:21 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:41:21.349923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50200eef-d3ec-42c6-bfd9-6220f1bfe93b None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:21.351432 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-50200eef-d3ec-42c6-bfd9-6220f1bfe93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:21.487741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bef6e-ab40-4699-a5bb-e1327f050605 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:21.487932 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bef6e-ab40-4699-a5bb-e1327f050605 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:21.488238 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bef6e-ab40-4699-a5bb-e1327f050605 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:21.489576 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 541/1077] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:41:21.494607 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cb0de-94e9-4263-8f05-9e833f03057c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:21.494911 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cb0de-94e9-4263-8f05-9e833f03057c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:21.495167 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cb0de-94e9-4263-8f05-9e833f03057c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:21.498381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50200eef-d3ec-42c6-bfd9-6220f1bfe93b None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:21.541642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:21.563463 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-e26cb0de-94e9-4263-8f05-9e833f03057c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:21.564715 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e26cb0de-94e9-4263-8f05-9e833f03057c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:21.605931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27bd293-91e6-4bd4-a8ec-7acb7689559e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f4c779b8422432fb06cdf996841b9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:21.606566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27bd293-91e6-4bd4-a8ec-7acb7689559e service swift] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:21.606928 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27bd293-91e6-4bd4-a8ec-7acb7689559e service swift] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:21.607115 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27bd293-91e6-4bd4-a8ec-7acb7689559e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:21.609285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c27bd293-91e6-4bd4-a8ec-7acb7689559e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:21.622815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:21.672249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 542/1078] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:41:21.699922 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27bd293-91e6-4bd4-a8ec-7acb7689559e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:21.702094 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27bd293-91e6-4bd4-a8ec-7acb7689559e service swift] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:21.711792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 537/1079] 10.222.0.36 () {60 vars in 1384 bytes} [Sun Oct 30 00:41:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 00:41:21.966852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cb0de-94e9-4263-8f05-9e833f03057c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:21.969111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e26cb0de-94e9-4263-8f05-9e833f03057c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:22.096919 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cb0de-94e9-4263-8f05-9e833f03057c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:22.181858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:22.235984 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c69ddc-2c35-434b-b34d-16458fb9457d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:22.236618 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c69ddc-2c35-434b-b34d-16458fb9457d service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:22.236897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c69ddc-2c35-434b-b34d-16458fb9457d service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:22.237279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c69ddc-2c35-434b-b34d-16458fb9457d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:22.239012 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c7c69ddc-2c35-434b-b34d-16458fb9457d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:22.347961 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c69ddc-2c35-434b-b34d-16458fb9457d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:22.349749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c69ddc-2c35-434b-b34d-16458fb9457d service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:22.369485 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 543/1080] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:41:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:41:24.605893 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b73ba-e0da-41c8-8756-7d6f34a20d81 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:24.606445 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b73ba-e0da-41c8-8756-7d6f34a20d81 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:24.606445 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b73ba-e0da-41c8-8756-7d6f34a20d81 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:24.608535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 538/1081] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:41:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:41:24.613809 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ebaef-126f-486e-a5c5-fc80a9967f60 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:24.614304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ebaef-126f-486e-a5c5-fc80a9967f60 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:24.614444 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ebaef-126f-486e-a5c5-fc80a9967f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:24.696739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-d83ebaef-126f-486e-a5c5-fc80a9967f60 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:24.698464 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d83ebaef-126f-486e-a5c5-fc80a9967f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:24.738136 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:24.820439 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 544/1082] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:41:25.112207 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b73ba-e0da-41c8-8756-7d6f34a20d81 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:25.113912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-b12b73ba-e0da-41c8-8756-7d6f34a20d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:25.200220 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:25.275737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b73ba-e0da-41c8-8756-7d6f34a20d81 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:25.427165 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eca06a-7fb4-49ff-98f2-eb1aeb553258 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:25.429483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eca06a-7fb4-49ff-98f2-eb1aeb553258 service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:25.430047 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eca06a-7fb4-49ff-98f2-eb1aeb553258 service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:25.430482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eca06a-7fb4-49ff-98f2-eb1aeb553258 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:25.434530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-36eca06a-7fb4-49ff-98f2-eb1aeb553258 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:25.509262 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:25.567038 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eca06a-7fb4-49ff-98f2-eb1aeb553258 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:25.568684 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eca06a-7fb4-49ff-98f2-eb1aeb553258 service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:25.588889 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 539/1083] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:41:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:41:27.811774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007cd3db-aea5-4410-86ab-6768d9413be2 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:27.812422 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007cd3db-aea5-4410-86ab-6768d9413be2 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:27.812422 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007cd3db-aea5-4410-86ab-6768d9413be2 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:27.814121 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 545/1084] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:41:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:27.819649 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4149df-a4ea-4444-9864-8773637e6ad1 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:27.820254 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4149df-a4ea-4444-9864-8773637e6ad1 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:27.820254 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4149df-a4ea-4444-9864-8773637e6ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:27.908591 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-3a4149df-a4ea-4444-9864-8773637e6ad1 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:27.909993 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-3a4149df-a4ea-4444-9864-8773637e6ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:28.028135 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 540/1085] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:41:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:41:28.316425 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007cd3db-aea5-4410-86ab-6768d9413be2 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:28.317932 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-007cd3db-aea5-4410-86ab-6768d9413be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:28.484380 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007cd3db-aea5-4410-86ab-6768d9413be2 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:28.504213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:28.614782 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ec94b-fd4d-4f86-94fb-5c355946431b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:28.615303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ec94b-fd4d-4f86-94fb-5c355946431b service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:28.615578 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ec94b-fd4d-4f86-94fb-5c355946431b service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:28.615845 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ec94b-fd4d-4f86-94fb-5c355946431b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:28.617472 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-7e7ec94b-fd4d-4f86-94fb-5c355946431b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:28.716194 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ec94b-fd4d-4f86-94fb-5c355946431b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:28.717845 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ec94b-fd4d-4f86-94fb-5c355946431b service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:28.736505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 546/1086] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:41:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:41:38.023896 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=94 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:38.734129 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=91 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:44.883021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707adbc-fd52-4e46-9f73-76b2bd11e957 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:44.883462 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707adbc-fd52-4e46-9f73-76b2bd11e957 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:44.883637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707adbc-fd52-4e46-9f73-76b2bd11e957 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:44.885181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 541/1087] 10.222.0.36 () {54 vars in 918 bytes} [Sun Oct 30 00:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:44.890573 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b014a-dee3-4bbf-8166-934916bb4fed None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:44.891361 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b014a-dee3-4bbf-8166-934916bb4fed None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:44.891638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b014a-dee3-4bbf-8166-934916bb4fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:44.973315 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-369b014a-dee3-4bbf-8166-934916bb4fed None None] MFA Rules not processed for user `5ded89d48d6c4a15a5c5829f594f8776`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:44.979820 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-369b014a-dee3-4bbf-8166-934916bb4fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:45.104380 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 547/1088] 10.222.0.36 () {58 vars in 1015 bytes} [Sun Oct 30 00:41:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:41:45.121496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87487111-0715-48a6-8661-909303e80cd5 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:45.121776 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87487111-0715-48a6-8661-909303e80cd5 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:45.122053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87487111-0715-48a6-8661-909303e80cd5 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:45.123388 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 542/1089] 10.222.0.36 () {54 vars in 918 bytes} [Sun Oct 30 00:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:45.128508 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ad132-5f22-4f0b-bb59-6d82b0ac634a None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:45.128762 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ad132-5f22-4f0b-bb59-6d82b0ac634a None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:45.129037 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ad132-5f22-4f0b-bb59-6d82b0ac634a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:45.213233 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:45.215931 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-6b5ad132-5f22-4f0b-bb59-6d82b0ac634a None None] MFA Rules not processed for user `adbba7be39a64230bab160ce77ce7586`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:45.217719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-6b5ad132-5f22-4f0b-bb59-6d82b0ac634a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:45.353195 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 548/1090] 10.222.0.36 () {58 vars in 1015 bytes} [Sun Oct 30 00:41:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:41:48.060718 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce004a-d104-4063-90ea-d32d4727c8c3 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:48.061248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce004a-d104-4063-90ea-d32d4727c8c3 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:48.061248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce004a-d104-4063-90ea-d32d4727c8c3 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:48.063261 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 543/1091] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 00:41:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:48.068665 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025da4fe-88d7-43f0-8713-fe54c007deab None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:48.069108 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025da4fe-88d7-43f0-8713-fe54c007deab None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:48.069239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025da4fe-88d7-43f0-8713-fe54c007deab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:48.134881 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-025da4fe-88d7-43f0-8713-fe54c007deab None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:48.136074 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-025da4fe-88d7-43f0-8713-fe54c007deab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:48.191316 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:48.233007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 549/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 00:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:41:48.528752 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce004a-d104-4063-90ea-d32d4727c8c3 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:48.530268 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-bbce004a-d104-4063-90ea-d32d4727c8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:48.676367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce004a-d104-4063-90ea-d32d4727c8c3 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:48.749202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:48.789961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4760309-e08e-4334-9663-d78371b09499 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:48.790597 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760309-e08e-4334-9663-d78371b09499 service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:48.790880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760309-e08e-4334-9663-d78371b09499 service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:48.791184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4760309-e08e-4334-9663-d78371b09499 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:48.792816 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e4760309-e08e-4334-9663-d78371b09499 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:48.883992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4760309-e08e-4334-9663-d78371b09499 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:48.885623 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4760309-e08e-4334-9663-d78371b09499 service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:48.905614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 544/1093] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:41:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:41:50.040051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d91a5-b3a1-49c0-9bff-ddfef9c441e7 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:50.040629 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d91a5-b3a1-49c0-9bff-ddfef9c441e7 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:50.040629 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d91a5-b3a1-49c0-9bff-ddfef9c441e7 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:50.041992 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 550/1094] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 00:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:50.047370 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded59b2-1aa2-4d79-820e-17da3a7d2860 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:50.047800 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded59b2-1aa2-4d79-820e-17da3a7d2860 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:50.047930 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded59b2-1aa2-4d79-820e-17da3a7d2860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:50.124248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-4ded59b2-1aa2-4d79-820e-17da3a7d2860 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:50.125511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4ded59b2-1aa2-4d79-820e-17da3a7d2860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:50.152540 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:50.223010 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 545/1095] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 00:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:41:50.383257 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d91a5-b3a1-49c0-9bff-ddfef9c441e7 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:50.384729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-263d91a5-b3a1-49c0-9bff-ddfef9c441e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:50.533691 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d91a5-b3a1-49c0-9bff-ddfef9c441e7 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:50.574830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:50.651283 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507d18a-58b7-4d6e-af27-818897deb95b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:50.651882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507d18a-58b7-4d6e-af27-818897deb95b service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:50.652189 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507d18a-58b7-4d6e-af27-818897deb95b service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:50.652461 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507d18a-58b7-4d6e-af27-818897deb95b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:50.654031 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d507d18a-58b7-4d6e-af27-818897deb95b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:50.746703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507d18a-58b7-4d6e-af27-818897deb95b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:50.748343 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507d18a-58b7-4d6e-af27-818897deb95b service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:50.767100 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 551/1096] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:41:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:41:51.923377 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de36ec9-4566-4be8-b3e1-fe86116af8a4 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:51.923912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de36ec9-4566-4be8-b3e1-fe86116af8a4 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:51.923912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de36ec9-4566-4be8-b3e1-fe86116af8a4 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:51.925225 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 546/1097] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 00:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:51.930099 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcfd1e-edb3-4c86-9886-7a0bae09d757 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:51.930545 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcfd1e-edb3-4c86-9886-7a0bae09d757 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:51.930654 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcfd1e-edb3-4c86-9886-7a0bae09d757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:51.985241 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:52.009128 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-28dcfd1e-edb3-4c86-9886-7a0bae09d757 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:52.010329 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-28dcfd1e-edb3-4c86-9886-7a0bae09d757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:52.107792 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 552/1098] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 00:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:41:52.259683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de36ec9-4566-4be8-b3e1-fe86116af8a4 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:52.261187 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-6de36ec9-4566-4be8-b3e1-fe86116af8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:52.404147 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de36ec9-4566-4be8-b3e1-fe86116af8a4 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:52.416290 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:52.518235 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfe9e5-0256-4930-8c39-03e2c1f6121a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:52.518780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfe9e5-0256-4930-8c39-03e2c1f6121a service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:52.519057 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfe9e5-0256-4930-8c39-03e2c1f6121a service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:52.519327 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfe9e5-0256-4930-8c39-03e2c1f6121a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:52.520950 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a8dfe9e5-0256-4930-8c39-03e2c1f6121a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:52.610864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dfe9e5-0256-4930-8c39-03e2c1f6121a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:52.612476 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dfe9e5-0256-4930-8c39-03e2c1f6121a service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:52.631927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 547/1099] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:41:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:41:53.776731 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf030f7-243e-4155-b252-2fe9b9e9f40b None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:53.777322 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf030f7-243e-4155-b252-2fe9b9e9f40b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:53.777322 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf030f7-243e-4155-b252-2fe9b9e9f40b None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:53.778705 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 553/1100] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 00:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:53.784617 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ea63a-e107-45bd-aa15-5401797c587f None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:53.785029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ea63a-e107-45bd-aa15-5401797c587f None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:53.785161 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ea63a-e107-45bd-aa15-5401797c587f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:53.805349 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:53.862485 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-dc3ea63a-e107-45bd-aa15-5401797c587f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:53.863679 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-dc3ea63a-e107-45bd-aa15-5401797c587f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:53.961845 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 548/1101] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 00:41:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:41:54.112312 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf030f7-243e-4155-b252-2fe9b9e9f40b None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:54.113773 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4bf030f7-243e-4155-b252-2fe9b9e9f40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:54.225712 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:54.262123 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf030f7-243e-4155-b252-2fe9b9e9f40b None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:54.374936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f6d0a-2d85-4e8f-b556-6f5a6c078746 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:54.375627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f6d0a-2d85-4e8f-b556-6f5a6c078746 service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:54.375925 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f6d0a-2d85-4e8f-b556-6f5a6c078746 service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:54.376233 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f6d0a-2d85-4e8f-b556-6f5a6c078746 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:54.377829 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a56f6d0a-2d85-4e8f-b556-6f5a6c078746 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:54.455458 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:54.469786 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f6d0a-2d85-4e8f-b556-6f5a6c078746 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:54.471439 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f6d0a-2d85-4e8f-b556-6f5a6c078746 service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:54.489620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 554/1102] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:41:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:41:55.600139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86328bdf-57a3-4a5b-93f2-5e343e02461a None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:55.600674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86328bdf-57a3-4a5b-93f2-5e343e02461a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:55.600674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86328bdf-57a3-4a5b-93f2-5e343e02461a None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:55.602037 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 549/1103] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 00:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:55.606967 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488de51-5536-4908-b261-6ebbf92c5470 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:55.607414 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488de51-5536-4908-b261-6ebbf92c5470 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:55.607513 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488de51-5536-4908-b261-6ebbf92c5470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:55.686094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-6488de51-5536-4908-b261-6ebbf92c5470 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:55.687383 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-6488de51-5536-4908-b261-6ebbf92c5470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:55.789992 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 555/1104] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 00:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:41:55.942894 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86328bdf-57a3-4a5b-93f2-5e343e02461a None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:55.944491 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-86328bdf-57a3-4a5b-93f2-5e343e02461a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:56.029667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:56.091685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86328bdf-57a3-4a5b-93f2-5e343e02461a None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:56.204303 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80150b5-f558-458f-8692-96931743c964 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:56.204902 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80150b5-f558-458f-8692-96931743c964 service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:56.205175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80150b5-f558-458f-8692-96931743c964 service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:56.205442 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80150b5-f558-458f-8692-96931743c964 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:56.207011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a80150b5-f558-458f-8692-96931743c964 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:56.257439 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:56.299380 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80150b5-f558-458f-8692-96931743c964 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:56.300992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80150b5-f558-458f-8692-96931743c964 service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:56.319059 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 550/1105] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:41:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:41:57.367916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb57e9-959c-401a-89d1-1bdedbb6e4e6 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:57.368550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb57e9-959c-401a-89d1-1bdedbb6e4e6 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:57.368550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb57e9-959c-401a-89d1-1bdedbb6e4e6 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:57.369865 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 556/1106] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 00:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:57.374678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dea37d-41c1-4a96-abfa-6c60acb47a5f None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:57.375620 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dea37d-41c1-4a96-abfa-6c60acb47a5f None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:57.376242 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dea37d-41c1-4a96-abfa-6c60acb47a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:57.453009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-87dea37d-41c1-4a96-abfa-6c60acb47a5f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:57.454240 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-87dea37d-41c1-4a96-abfa-6c60acb47a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:57.551331 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 551/1107] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 00:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:41:57.700760 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb57e9-959c-401a-89d1-1bdedbb6e4e6 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:57.702264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-37fb57e9-959c-401a-89d1-1bdedbb6e4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:57.752921 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:57.850379 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb57e9-959c-401a-89d1-1bdedbb6e4e6 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:57.967251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3297f-ed3b-4264-b0e2-dc7d599e3c50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:57.967799 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3297f-ed3b-4264-b0e2-dc7d599e3c50 service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:57.968105 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3297f-ed3b-4264-b0e2-dc7d599e3c50 service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:57.968381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3297f-ed3b-4264-b0e2-dc7d599e3c50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:57.970660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-7ae3297f-ed3b-4264-b0e2-dc7d599e3c50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:57.978124 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:58.070308 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3297f-ed3b-4264-b0e2-dc7d599e3c50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:58.071915 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3297f-ed3b-4264-b0e2-dc7d599e3c50 service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:58.089788 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 557/1108] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:41:59.139004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cacd7-d0a8-4a14-8ee9-6c0748a941ea None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:59.139540 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cacd7-d0a8-4a14-8ee9-6c0748a941ea None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:59.139540 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cacd7-d0a8-4a14-8ee9-6c0748a941ea None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:59.141439 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 552/1109] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 00:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:41:59.146437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0a4ca-5704-41e8-9ac3-9a0f84963ae7 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:59.146856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0a4ca-5704-41e8-9ac3-9a0f84963ae7 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:59.146970 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0a4ca-5704-41e8-9ac3-9a0f84963ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:59.223635 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-e8a0a4ca-5704-41e8-9ac3-9a0f84963ae7 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:41:59.224880 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e8a0a4ca-5704-41e8-9ac3-9a0f84963ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:59.291830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:59.322167 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 558/1110] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 00:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:41:59.473843 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cacd7-d0a8-4a14-8ee9-6c0748a941ea None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:41:59.475412 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-8b2cacd7-d0a8-4a14-8ee9-6c0748a941ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:59.493001 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:59.622122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cacd7-d0a8-4a14-8ee9-6c0748a941ea None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:41:59.708417 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:41:59.733688 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74406aae-d6c9-400a-9c08-83c056bb4563 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:41:59.734220 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74406aae-d6c9-400a-9c08-83c056bb4563 service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:41:59.735162 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74406aae-d6c9-400a-9c08-83c056bb4563 service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:41:59.735394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74406aae-d6c9-400a-9c08-83c056bb4563 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:41:59.736967 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-74406aae-d6c9-400a-9c08-83c056bb4563 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:41:59.827061 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74406aae-d6c9-400a-9c08-83c056bb4563 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:41:59.829185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74406aae-d6c9-400a-9c08-83c056bb4563 service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:41:59.848784 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 553/1111] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:41:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:42:00.896479 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d665f30-1247-4bbc-90fc-7d0819a22e60 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:00.897037 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d665f30-1247-4bbc-90fc-7d0819a22e60 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:00.897037 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d665f30-1247-4bbc-90fc-7d0819a22e60 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:00.898382 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 559/1112] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 00:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:42:00.903552 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d417c-b6a3-4d18-bbb8-763e268bd3eb None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:00.903969 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d417c-b6a3-4d18-bbb8-763e268bd3eb None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:00.904137 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d417c-b6a3-4d18-bbb8-763e268bd3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:00.979380 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-a49d417c-b6a3-4d18-bbb8-763e268bd3eb None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:42:00.980608 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a49d417c-b6a3-4d18-bbb8-763e268bd3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:01.022367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:01.077289 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 554/1113] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 00:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:42:01.229378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d665f30-1247-4bbc-90fc-7d0819a22e60 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:42:01.230925 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-2d665f30-1247-4bbc-90fc-7d0819a22e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:01.377807 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d665f30-1247-4bbc-90fc-7d0819a22e60 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:42:01.434221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:01.494049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3eedd8-0542-4919-993b-bdb49766bc5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:42:01.494582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eedd8-0542-4919-993b-bdb49766bc5f service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:01.494858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eedd8-0542-4919-993b-bdb49766bc5f service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:01.495124 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eedd8-0542-4919-993b-bdb49766bc5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:01.496755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-0f3eedd8-0542-4919-993b-bdb49766bc5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:01.585270 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3eedd8-0542-4919-993b-bdb49766bc5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:42:01.586868 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3eedd8-0542-4919-993b-bdb49766bc5f service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:42:01.605135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 560/1114] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:42:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:42:02.647029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470864b-57a5-4a90-910d-1613816ab789 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:02.647610 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470864b-57a5-4a90-910d-1613816ab789 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:02.647610 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470864b-57a5-4a90-910d-1613816ab789 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:02.648965 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 555/1115] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 00:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:42:02.653910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373e670-1587-4d4b-946a-a760510409d6 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:02.654338 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373e670-1587-4d4b-946a-a760510409d6 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:02.654456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373e670-1587-4d4b-946a-a760510409d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:02.731446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-d373e670-1587-4d4b-946a-a760510409d6 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:42:02.732595 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d373e670-1587-4d4b-946a-a760510409d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:02.734575 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:02.829079 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 561/1116] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 00:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:42:02.980548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470864b-57a5-4a90-910d-1613816ab789 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:42:02.982144 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-9470864b-57a5-4a90-910d-1613816ab789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:03.127250 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470864b-57a5-4a90-910d-1613816ab789 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:42:03.154391 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:03.243809 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa34a1-c2f6-444d-918d-6590bc68c5b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:42:03.244358 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa34a1-c2f6-444d-918d-6590bc68c5b8 service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:03.244632 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa34a1-c2f6-444d-918d-6590bc68c5b8 service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:03.244895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa34a1-c2f6-444d-918d-6590bc68c5b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:03.246465 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-cdaa34a1-c2f6-444d-918d-6590bc68c5b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:03.336589 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaa34a1-c2f6-444d-918d-6590bc68c5b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:42:03.338175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaa34a1-c2f6-444d-918d-6590bc68c5b8 service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:42:03.355896 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 556/1117] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:42:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:42:05.484750 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d79c5-a167-474c-8f8a-b782463f4783 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:05.485431 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d79c5-a167-474c-8f8a-b782463f4783 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:05.486262 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d79c5-a167-474c-8f8a-b782463f4783 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:05.487659 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 562/1118] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 00:42:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:42:05.493295 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644ccd8-4d95-49fd-97e2-8e6ca3dfa3d3 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:05.493731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644ccd8-4d95-49fd-97e2-8e6ca3dfa3d3 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:05.493846 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644ccd8-4d95-49fd-97e2-8e6ca3dfa3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:05.534330 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:05.560213 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-a644ccd8-4d95-49fd-97e2-8e6ca3dfa3d3 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:42:05.561426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a644ccd8-4d95-49fd-97e2-8e6ca3dfa3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:05.658226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 557/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 00:42:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:42:05.944241 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d79c5-a167-474c-8f8a-b782463f4783 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:42:05.945756 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b39d79c5-a167-474c-8f8a-b782463f4783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:06.048433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d79c5-a167-474c-8f8a-b782463f4783 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:42:06.081612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:06.128666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936a138-96aa-4874-b264-faaef18be9bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:42:06.129265 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936a138-96aa-4874-b264-faaef18be9bb service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:06.129538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936a138-96aa-4874-b264-faaef18be9bb service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:06.129804 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936a138-96aa-4874-b264-faaef18be9bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:06.131382 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-5936a138-96aa-4874-b264-faaef18be9bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:06.238077 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5936a138-96aa-4874-b264-faaef18be9bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:42:06.239511 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5936a138-96aa-4874-b264-faaef18be9bb service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:42:06.257581 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 563/1120] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:42:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:42:08.650870 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fc743-2f42-4915-803d-d22544e861ae None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:08.651430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fc743-2f42-4915-803d-d22544e861ae None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:08.651430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fc743-2f42-4915-803d-d22544e861ae None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:08.652879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 558/1121] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 00:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:42:08.658260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601faa4f-0b2c-418c-8b7a-5c9ad19ecaa2 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:08.658659 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601faa4f-0b2c-418c-8b7a-5c9ad19ecaa2 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:08.658813 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601faa4f-0b2c-418c-8b7a-5c9ad19ecaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:08.725053 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-601faa4f-0b2c-418c-8b7a-5c9ad19ecaa2 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:42:08.726259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-601faa4f-0b2c-418c-8b7a-5c9ad19ecaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:08.739718 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:08.826597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 564/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 00:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:42:09.116530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fc743-2f42-4915-803d-d22544e861ae None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:42:09.118091 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-584fc743-2f42-4915-803d-d22544e861ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:09.220751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fc743-2f42-4915-803d-d22544e861ae None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:42:09.233405 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:09.295769 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d1b0f-fc56-4ffd-a9ba-ef8e9e81003b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:42:09.296411 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d1b0f-fc56-4ffd-a9ba-ef8e9e81003b service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:09.296693 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d1b0f-fc56-4ffd-a9ba-ef8e9e81003b service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:09.296967 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d1b0f-fc56-4ffd-a9ba-ef8e9e81003b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:09.298642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-cb5d1b0f-fc56-4ffd-a9ba-ef8e9e81003b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:09.407577 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d1b0f-fc56-4ffd-a9ba-ef8e9e81003b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:42:09.409036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d1b0f-fc56-4ffd-a9ba-ef8e9e81003b service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:42:09.427395 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 559/1123] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:42:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:42:11.904807 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8d955-b1de-4041-867a-479c35170485 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:11.905351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8d955-b1de-4041-867a-479c35170485 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:11.905351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8d955-b1de-4041-867a-479c35170485 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:11.906713 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 565/1124] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 00:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:42:11.912777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60eeb39-8c8a-4c78-82d7-49cac9bbb0a0 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:11.913225 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60eeb39-8c8a-4c78-82d7-49cac9bbb0a0 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:11.913328 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60eeb39-8c8a-4c78-82d7-49cac9bbb0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:11.961284 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:11.979419 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-d60eeb39-8c8a-4c78-82d7-49cac9bbb0a0 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:42:11.980659 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d60eeb39-8c8a-4c78-82d7-49cac9bbb0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:12.076032 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 560/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 00:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:42:12.368803 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8d955-b1de-4041-867a-479c35170485 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:42:12.370349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-61b8d955-b1de-4041-867a-479c35170485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:12.475695 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8d955-b1de-4041-867a-479c35170485 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:42:12.515126 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:12.549701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603d56d-eea9-47fb-b933-40a6281617ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:42:12.550302 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603d56d-eea9-47fb-b933-40a6281617ad service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:12.550577 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603d56d-eea9-47fb-b933-40a6281617ad service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:12.550846 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603d56d-eea9-47fb-b933-40a6281617ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:12.552447 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-1603d56d-eea9-47fb-b933-40a6281617ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:12.660518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603d56d-eea9-47fb-b933-40a6281617ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:42:12.661958 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603d56d-eea9-47fb-b933-40a6281617ad service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:42:12.679492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 566/1126] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:42:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:42:15.003785 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957f267-eec1-4abe-b648-233de4d0f31b None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:15.004376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957f267-eec1-4abe-b648-233de4d0f31b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:15.004376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957f267-eec1-4abe-b648-233de4d0f31b None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:15.005744 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 561/1127] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 00:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:42:15.011149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db84bcc-280f-4d12-b39b-e2c54ac02987 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:15.011578 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db84bcc-280f-4d12-b39b-e2c54ac02987 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:15.011698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db84bcc-280f-4d12-b39b-e2c54ac02987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:15.078032 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-3db84bcc-280f-4d12-b39b-e2c54ac02987 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:42:15.079342 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-3db84bcc-280f-4d12-b39b-e2c54ac02987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:15.100311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:15.179498 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 567/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 00:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:42:15.465628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957f267-eec1-4abe-b648-233de4d0f31b None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:42:15.467266 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-7957f267-eec1-4abe-b648-233de4d0f31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:15.571988 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957f267-eec1-4abe-b648-233de4d0f31b None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:42:15.592301 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:15.646006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2cba0-6282-4415-8670-01f02da784ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:42:15.646603 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2cba0-6282-4415-8670-01f02da784ec service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:15.646884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2cba0-6282-4415-8670-01f02da784ec service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:15.647151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2cba0-6282-4415-8670-01f02da784ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:15.648741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d1c2cba0-6282-4415-8670-01f02da784ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:15.756883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2cba0-6282-4415-8670-01f02da784ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:42:15.758227 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2cba0-6282-4415-8670-01f02da784ec service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:42:15.775904 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 562/1129] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:42:18.015178 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f16ec-e5af-46c4-ab0d-66bd2bd1ef05 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:18.015759 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f16ec-e5af-46c4-ab0d-66bd2bd1ef05 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:18.015759 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f16ec-e5af-46c4-ab0d-66bd2bd1ef05 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:18.017885 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 568/1130] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 00:42:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:42:18.023772 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243d358-90e0-48b7-a6b5-335dd69ca3b3 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:18.024280 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243d358-90e0-48b7-a6b5-335dd69ca3b3 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:18.024385 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243d358-90e0-48b7-a6b5-335dd69ca3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:18.088863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:18.091534 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-9243d358-90e0-48b7-a6b5-335dd69ca3b3 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:42:18.093376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-9243d358-90e0-48b7-a6b5-335dd69ca3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:18.191077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 563/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 00:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:42:18.478309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f16ec-e5af-46c4-ab0d-66bd2bd1ef05 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:42:18.479855 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-cf7f16ec-e5af-46c4-ab0d-66bd2bd1ef05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:18.584653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f16ec-e5af-46c4-ab0d-66bd2bd1ef05 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:42:18.634993 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:18.659632 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ca2d4-46f5-41c8-b870-1f314150965f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:42:18.660273 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ca2d4-46f5-41c8-b870-1f314150965f service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:18.660554 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ca2d4-46f5-41c8-b870-1f314150965f service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:18.660828 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ca2d4-46f5-41c8-b870-1f314150965f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:18.662436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-415ca2d4-46f5-41c8-b870-1f314150965f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:18.771991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ca2d4-46f5-41c8-b870-1f314150965f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:42:18.773376 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ca2d4-46f5-41c8-b870-1f314150965f service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:42:18.791526 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 569/1132] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:42:21.020353 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14881cb0-d864-48fd-b8dc-abab06aa38f7 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:21.020890 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14881cb0-d864-48fd-b8dc-abab06aa38f7 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:21.020890 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14881cb0-d864-48fd-b8dc-abab06aa38f7 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:21.022248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 564/1133] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 00:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:42:21.027624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe07c7a-6640-47cd-8ab5-664fcf8e50c0 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:21.028013 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe07c7a-6640-47cd-8ab5-664fcf8e50c0 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:21.028322 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe07c7a-6640-47cd-8ab5-664fcf8e50c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:21.093889 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-dbe07c7a-6640-47cd-8ab5-664fcf8e50c0 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:42:21.095109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-dbe07c7a-6640-47cd-8ab5-664fcf8e50c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:21.123292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:21.198462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 570/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 00:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:42:21.483222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14881cb0-d864-48fd-b8dc-abab06aa38f7 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:42:21.484790 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-14881cb0-d864-48fd-b8dc-abab06aa38f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:21.589454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14881cb0-d864-48fd-b8dc-abab06aa38f7 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:42:21.619696 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:21.665207 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413fc92f-60c0-4d97-8673-994efe167689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:42:21.665865 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fc92f-60c0-4d97-8673-994efe167689 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:21.669817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fc92f-60c0-4d97-8673-994efe167689 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:21.670067 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fc92f-60c0-4d97-8673-994efe167689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:21.671743 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-413fc92f-60c0-4d97-8673-994efe167689 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:21.784948 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413fc92f-60c0-4d97-8673-994efe167689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:42:21.786278 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413fc92f-60c0-4d97-8673-994efe167689 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:42:21.805011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 565/1135] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:42:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:42:24.074238 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a0449-ebdf-4e4f-899e-f0ff1a1e0054 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:24.074749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a0449-ebdf-4e4f-899e-f0ff1a1e0054 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:24.074749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a0449-ebdf-4e4f-899e-f0ff1a1e0054 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:24.076042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 571/1136] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 00:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:42:24.081372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40865a7-de0b-464a-b70d-f425bae3edd8 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:24.081800 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40865a7-de0b-464a-b70d-f425bae3edd8 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:24.081913 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40865a7-de0b-464a-b70d-f425bae3edd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:24.148474 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-c40865a7-de0b-464a-b70d-f425bae3edd8 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:42:24.149722 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c40865a7-de0b-464a-b70d-f425bae3edd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:24.158584 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:24.248092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 566/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 00:42:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:42:24.540225 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a0449-ebdf-4e4f-899e-f0ff1a1e0054 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:42:24.541752 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b93a0449-ebdf-4e4f-899e-f0ff1a1e0054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:24.648512 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a0449-ebdf-4e4f-899e-f0ff1a1e0054 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:42:24.703482 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:24.723198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621cd6b9-947d-40bb-b338-c36c96debefb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:42:24.723793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cd6b9-947d-40bb-b338-c36c96debefb service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:24.724068 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cd6b9-947d-40bb-b338-c36c96debefb service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:24.724358 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cd6b9-947d-40bb-b338-c36c96debefb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:24.725928 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-621cd6b9-947d-40bb-b338-c36c96debefb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:24.838054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621cd6b9-947d-40bb-b338-c36c96debefb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:42:24.839493 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621cd6b9-947d-40bb-b338-c36c96debefb service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:42:24.857976 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 572/1138] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:42:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:42:27.083838 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2803f-eb5b-41ab-b396-4621ee265f45 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:27.084436 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2803f-eb5b-41ab-b396-4621ee265f45 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:27.084436 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2803f-eb5b-41ab-b396-4621ee265f45 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:27.085762 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 567/1139] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 00:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:42:27.090955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3962642-dd13-4887-874e-e3c32359be6c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:27.091365 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3962642-dd13-4887-874e-e3c32359be6c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:27.091502 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3962642-dd13-4887-874e-e3c32359be6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:27.159106 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-e3962642-dd13-4887-874e-e3c32359be6c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:42:27.160346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e3962642-dd13-4887-874e-e3c32359be6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:27.199308 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:27.261761 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 573/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 00:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:42:27.545027 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2803f-eb5b-41ab-b396-4621ee265f45 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:42:27.546567 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-3db2803f-eb5b-41ab-b396-4621ee265f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:27.650614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2803f-eb5b-41ab-b396-4621ee265f45 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:42:27.689050 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:27.725551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa260f2b-34e1-4796-afeb-fa05e01c15c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:42:27.726159 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa260f2b-34e1-4796-afeb-fa05e01c15c3 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:27.726435 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa260f2b-34e1-4796-afeb-fa05e01c15c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:27.726706 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa260f2b-34e1-4796-afeb-fa05e01c15c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:27.728338 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-aa260f2b-34e1-4796-afeb-fa05e01c15c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:27.837739 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa260f2b-34e1-4796-afeb-fa05e01c15c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:42:27.839083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa260f2b-34e1-4796-afeb-fa05e01c15c3 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:42:27.857104 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 568/1141] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:42:30.048431 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301b837-dd04-46a5-ba47-84cfae08cdad None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:30.049052 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301b837-dd04-46a5-ba47-84cfae08cdad None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:30.049052 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301b837-dd04-46a5-ba47-84cfae08cdad None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:30.050518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 574/1142] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 00:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:42:30.055902 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26132797-af49-4f47-a8be-afc69ee7facc None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:30.056332 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26132797-af49-4f47-a8be-afc69ee7facc None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:30.056478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26132797-af49-4f47-a8be-afc69ee7facc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:30.123309 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-26132797-af49-4f47-a8be-afc69ee7facc None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:42:30.124581 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-26132797-af49-4f47-a8be-afc69ee7facc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:30.144137 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:30.222841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 569/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 00:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:42:30.520278 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301b837-dd04-46a5-ba47-84cfae08cdad None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:42:30.522084 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-3301b837-dd04-46a5-ba47-84cfae08cdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:30.629495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301b837-dd04-46a5-ba47-84cfae08cdad None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:42:30.697338 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:30.704292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72daf59-6217-47ca-ae3b-140436342303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:42:30.705431 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72daf59-6217-47ca-ae3b-140436342303 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:30.705747 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72daf59-6217-47ca-ae3b-140436342303 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:30.706016 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72daf59-6217-47ca-ae3b-140436342303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:30.707649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-f72daf59-6217-47ca-ae3b-140436342303 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:30.817413 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72daf59-6217-47ca-ae3b-140436342303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:42:30.818784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72daf59-6217-47ca-ae3b-140436342303 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:42:30.837422 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 575/1144] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:42:33.011594 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417172c1-116a-448e-b811-c460b955fe74 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:33.012180 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417172c1-116a-448e-b811-c460b955fe74 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:33.012180 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417172c1-116a-448e-b811-c460b955fe74 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:33.013498 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 570/1145] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 00:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:42:33.019642 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e804afa-39f3-4c3b-b43f-1ab2a8cf08de None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:33.020130 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e804afa-39f3-4c3b-b43f-1ab2a8cf08de None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:33.020240 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e804afa-39f3-4c3b-b43f-1ab2a8cf08de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:33.087083 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-1e804afa-39f3-4c3b-b43f-1ab2a8cf08de None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:42:33.088321 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-1e804afa-39f3-4c3b-b43f-1ab2a8cf08de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:33.136010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:33.183597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 576/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 00:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:42:33.471694 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417172c1-116a-448e-b811-c460b955fe74 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:42:33.473284 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-417172c1-116a-448e-b811-c460b955fe74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:33.576623 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417172c1-116a-448e-b811-c460b955fe74 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:42:33.624889 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:33.651453 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738731f2-3e0a-4af6-914a-99669a0ff290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:42:33.652069 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738731f2-3e0a-4af6-914a-99669a0ff290 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:33.652365 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738731f2-3e0a-4af6-914a-99669a0ff290 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:33.652639 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738731f2-3e0a-4af6-914a-99669a0ff290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:33.654223 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-738731f2-3e0a-4af6-914a-99669a0ff290 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:33.765662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738731f2-3e0a-4af6-914a-99669a0ff290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:42:33.767012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738731f2-3e0a-4af6-914a-99669a0ff290 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:42:33.785653 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 571/1147] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:42:43.182082 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=22 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:43.783190 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=66 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:54.014373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9c49a-76df-4b42-b401-1ac842753948 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:54.014912 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9c49a-76df-4b42-b401-1ac842753948 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:54.014912 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9c49a-76df-4b42-b401-1ac842753948 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:54.016325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 577/1148] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:42:54.022251 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc2988-f937-41db-8cb6-e81aae8e816e None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:54.022708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc2988-f937-41db-8cb6-e81aae8e816e None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:54.022808 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc2988-f937-41db-8cb6-e81aae8e816e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:54.090320 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-97bc2988-f937-41db-8cb6-e81aae8e816e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:42:54.091515 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-97bc2988-f937-41db-8cb6-e81aae8e816e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:54.185762 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 572/1149] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:42:54.490371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe4b13-6060-419b-ae10-e57a07990996 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:54.490732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe4b13-6060-419b-ae10-e57a07990996 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:54.491017 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe4b13-6060-419b-ae10-e57a07990996 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:54.492386 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 578/1150] 10.222.0.36 () {54 vars in 962 bytes} [Sun Oct 30 00:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:42:54.497280 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24f845-c447-439a-a1c4-ec58420c3526 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:54.497589 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24f845-c447-439a-a1c4-ec58420c3526 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:54.497859 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24f845-c447-439a-a1c4-ec58420c3526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:54.580180 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-1e24f845-c447-439a-a1c4-ec58420c3526 None None] MFA Rules not processed for user `7cf9451f2ef7478589b4a4278cb03252`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:42:54.581462 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-1e24f845-c447-439a-a1c4-ec58420c3526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:54.601971 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:54.658592 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 573/1151] 10.222.0.36 () {58 vars in 1059 bytes} [Sun Oct 30 00:42:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:42:54.666539 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe4b13-6060-419b-ae10-e57a07990996 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:42:54.668061 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b2fe4b13-6060-419b-ae10-e57a07990996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:54.772248 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe4b13-6060-419b-ae10-e57a07990996 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:42:54.851874 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81872a-37e9-4f7f-a164-fc8866024087 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cf9451f2ef7478589b4a4278cb03252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:42:54.852515 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81872a-37e9-4f7f-a164-fc8866024087 service glance] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:54.852793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81872a-37e9-4f7f-a164-fc8866024087 service glance] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:54.853060 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81872a-37e9-4f7f-a164-fc8866024087 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:54.854684 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-9b81872a-37e9-4f7f-a164-fc8866024087 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:54.912889 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:54.967800 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81872a-37e9-4f7f-a164-fc8866024087 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:42:54.969289 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81872a-37e9-4f7f-a164-fc8866024087 service glance] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:42:54.987379 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 579/1152] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:42:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:42:57.228309 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b385b6-d884-4b57-b80c-ff3be23b9f93 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:57.228875 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b385b6-d884-4b57-b80c-ff3be23b9f93 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:57.228875 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b385b6-d884-4b57-b80c-ff3be23b9f93 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:57.230187 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 574/1153] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:42:57.235399 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fcb90-20b3-4568-9d81-89510959edd5 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:57.235828 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fcb90-20b3-4568-9d81-89510959edd5 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:57.235955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fcb90-20b3-4568-9d81-89510959edd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:57.302198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-8e6fcb90-20b3-4568-9d81-89510959edd5 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:42:57.303415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-8e6fcb90-20b3-4568-9d81-89510959edd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:57.399312 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 580/1154] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:42:57.676719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b385b6-d884-4b57-b80c-ff3be23b9f93 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:42:57.678239 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-99b385b6-d884-4b57-b80c-ff3be23b9f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:42:57.802859 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b385b6-d884-4b57-b80c-ff3be23b9f93 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:42:57.854865 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:42:57.900308 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5c356-57ca-45e5-9941-2f32953dd280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:42:57.900933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5c356-57ca-45e5-9941-2f32953dd280 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:42:57.901214 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5c356-57ca-45e5-9941-2f32953dd280 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:42:57.901483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5c356-57ca-45e5-9941-2f32953dd280 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:42:57.902120 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5c356-57ca-45e5-9941-2f32953dd280 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:42:57.903228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: WARNING py.warnings [None req-35d5c356-57ca-45e5-9941-2f32953dd280 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:42:57.903228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: warnings.warn(msg) Oct 30 00:42:57.903228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: Oct 30 00:42:57.904394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5c356-57ca-45e5-9941-2f32953dd280 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:42:57.923686 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 575/1155] 10.222.0.36 () {56 vars in 1108 bytes} [Sun Oct 30 00:42:57 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:43:00.135216 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6afd9-68b5-4e80-bb78-caeb1c7ef516 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:43:00.135783 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6afd9-68b5-4e80-bb78-caeb1c7ef516 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:43:00.135783 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6afd9-68b5-4e80-bb78-caeb1c7ef516 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:43:00.137076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 581/1156] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:43:00.142415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea5c06-1477-491b-8833-08d63b010403 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:43:00.142822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea5c06-1477-491b-8833-08d63b010403 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:43:00.142972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea5c06-1477-491b-8833-08d63b010403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:43:00.209407 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-08ea5c06-1477-491b-8833-08d63b010403 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:43:00.210612 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-08ea5c06-1477-491b-8833-08d63b010403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:43:00.304795 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 576/1157] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:43:00.610318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d6afd9-68b5-4e80-bb78-caeb1c7ef516 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:43:00.611819 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e9d6afd9-68b5-4e80-bb78-caeb1c7ef516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:43:00.622036 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:43:00.768138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d6afd9-68b5-4e80-bb78-caeb1c7ef516 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:43:00.848758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:43:00.883238 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f68d1-4384-4a3e-af17-7216bb3b5d98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:43:00.883840 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f68d1-4384-4a3e-af17-7216bb3b5d98 service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:43:00.884146 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f68d1-4384-4a3e-af17-7216bb3b5d98 service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:43:00.884414 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f68d1-4384-4a3e-af17-7216bb3b5d98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:43:00.885973 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-729f68d1-4384-4a3e-af17-7216bb3b5d98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:43:00.979385 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f68d1-4384-4a3e-af17-7216bb3b5d98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:43:00.980961 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f68d1-4384-4a3e-af17-7216bb3b5d98 service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:43:00.998585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 582/1158] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:43:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:43:03.399713 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8d50a-6f24-461d-b1e3-152b7f04e1af None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:43:03.400322 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8d50a-6f24-461d-b1e3-152b7f04e1af None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:43:03.400322 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8d50a-6f24-461d-b1e3-152b7f04e1af None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:43:03.401672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 577/1159] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:43:03.406862 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b808d-5f64-47d4-ac7d-66b37b08552e None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:43:03.407267 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b808d-5f64-47d4-ac7d-66b37b08552e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:43:03.407411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b808d-5f64-47d4-ac7d-66b37b08552e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:43:03.475694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-670b808d-5f64-47d4-ac7d-66b37b08552e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:43:03.476926 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-670b808d-5f64-47d4-ac7d-66b37b08552e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:43:03.524166 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:43:03.575656 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 583/1160] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:43:03.874594 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8d50a-6f24-461d-b1e3-152b7f04e1af None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:43:03.876138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-bcc8d50a-6f24-461d-b1e3-152b7f04e1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:43:03.901824 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:43:03.983081 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8d50a-6f24-461d-b1e3-152b7f04e1af None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:43:04.056443 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0204b7-e2f9-447b-9478-7a98c04fc3f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:43:04.057045 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0204b7-e2f9-447b-9478-7a98c04fc3f7 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:43:04.057323 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0204b7-e2f9-447b-9478-7a98c04fc3f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:43:04.057612 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0204b7-e2f9-447b-9478-7a98c04fc3f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:43:04.059183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-8d0204b7-e2f9-447b-9478-7a98c04fc3f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:43:04.135570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:43:04.170260 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0204b7-e2f9-447b-9478-7a98c04fc3f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:43:04.171700 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0204b7-e2f9-447b-9478-7a98c04fc3f7 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:43:04.190575 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 578/1161] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:43:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:43:06.438819 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e62286-37b2-4c18-8171-ce3bb10796d3 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:43:06.439388 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e62286-37b2-4c18-8171-ce3bb10796d3 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:43:06.439388 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e62286-37b2-4c18-8171-ce3bb10796d3 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:43:06.440788 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 584/1162] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:43:06.446745 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9141cd-eebc-4f33-8ca5-3eb976cf5e31 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:43:06.447172 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9141cd-eebc-4f33-8ca5-3eb976cf5e31 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:43:06.447272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9141cd-eebc-4f33-8ca5-3eb976cf5e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:43:06.513288 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-1a9141cd-eebc-4f33-8ca5-3eb976cf5e31 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:43:06.514481 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-1a9141cd-eebc-4f33-8ca5-3eb976cf5e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:43:06.612248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 579/1163] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:43:06.921011 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e62286-37b2-4c18-8171-ce3bb10796d3 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:43:06.922572 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c9e62286-37b2-4c18-8171-ce3bb10796d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:43:07.027512 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e62286-37b2-4c18-8171-ce3bb10796d3 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:43:07.102431 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:43:07.103943 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da1656-e7d4-4ee9-8f64-3f1507a38b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:43:07.104668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da1656-e7d4-4ee9-8f64-3f1507a38b56 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:43:07.105146 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da1656-e7d4-4ee9-8f64-3f1507a38b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:43:07.105650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da1656-e7d4-4ee9-8f64-3f1507a38b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:43:07.107488 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-10da1656-e7d4-4ee9-8f64-3f1507a38b56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:43:07.220173 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da1656-e7d4-4ee9-8f64-3f1507a38b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:43:07.221626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da1656-e7d4-4ee9-8f64-3f1507a38b56 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:43:07.239975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 585/1164] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:43:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:43:09.499096 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ca4a3-6233-4373-a997-926e0c991f8a None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:43:09.499665 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ca4a3-6233-4373-a997-926e0c991f8a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:43:09.499665 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ca4a3-6233-4373-a997-926e0c991f8a None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:43:09.501044 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 580/1165] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:43:09.507054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d01d6d-60e6-4eae-a8ee-32ec0630aeaf None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:43:09.507478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d01d6d-60e6-4eae-a8ee-32ec0630aeaf None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:43:09.507588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d01d6d-60e6-4eae-a8ee-32ec0630aeaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:43:09.574833 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-66d01d6d-60e6-4eae-a8ee-32ec0630aeaf None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:43:09.576058 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-66d01d6d-60e6-4eae-a8ee-32ec0630aeaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:43:09.628687 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:43:09.677960 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 586/1166] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:43:09.976700 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ca4a3-6233-4373-a997-926e0c991f8a None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:43:09.978260 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-fb6ca4a3-6233-4373-a997-926e0c991f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:43:10.014363 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:43:10.081942 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ca4a3-6233-4373-a997-926e0c991f8a None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:43:10.155741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de07419-864a-4cc8-830a-905404204a4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:43:10.156370 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de07419-864a-4cc8-830a-905404204a4f service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:43:10.156651 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de07419-864a-4cc8-830a-905404204a4f service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:43:10.156924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de07419-864a-4cc8-830a-905404204a4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:43:10.158492 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-1de07419-864a-4cc8-830a-905404204a4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:43:10.244098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:43:10.275514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de07419-864a-4cc8-830a-905404204a4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:43:10.277005 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de07419-864a-4cc8-830a-905404204a4f service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:43:10.295691 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 581/1167] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:43:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:43:12.684611 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ea280-0ffe-4dc9-bd69-e9c618836773 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:43:12.685174 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ea280-0ffe-4dc9-bd69-e9c618836773 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:43:12.685174 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ea280-0ffe-4dc9-bd69-e9c618836773 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:43:12.686533 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 587/1168] 10.222.0.36 () {54 vars in 927 bytes} [Sun Oct 30 00:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:43:12.693042 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40efb9-00ca-4c8d-8687-14f5f8af1e31 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:43:12.693042 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40efb9-00ca-4c8d-8687-14f5f8af1e31 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:43:12.693581 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40efb9-00ca-4c8d-8687-14f5f8af1e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:43:12.758611 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-4e40efb9-00ca-4c8d-8687-14f5f8af1e31 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:43:12.759828 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4e40efb9-00ca-4c8d-8687-14f5f8af1e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:43:12.858135 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 582/1169] 10.222.0.36 () {58 vars in 1024 bytes} [Sun Oct 30 00:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:43:13.149232 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ea280-0ffe-4dc9-bd69-e9c618836773 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:43:13.150820 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-655ea280-0ffe-4dc9-bd69-e9c618836773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:43:13.252759 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ea280-0ffe-4dc9-bd69-e9c618836773 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:43:13.328059 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddded4e-9e5d-4455-a3f9-bb578003be95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:43:13.328687 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddded4e-9e5d-4455-a3f9-bb578003be95 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:43:13.328962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddded4e-9e5d-4455-a3f9-bb578003be95 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:43:13.329228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddded4e-9e5d-4455-a3f9-bb578003be95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:43:13.330787 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-cddded4e-9e5d-4455-a3f9-bb578003be95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:43:13.340034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:43:13.441607 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddded4e-9e5d-4455-a3f9-bb578003be95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:43:13.443625 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddded4e-9e5d-4455-a3f9-bb578003be95 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:43:13.461327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 588/1170] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:43:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:43:22.856597 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=82 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:43:23.459195 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=50 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:44:33.982594 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9071b9-35e1-4345-9694-2941291bd5e7 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:44:33.983159 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9071b9-35e1-4345-9694-2941291bd5e7 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:44:33.983159 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9071b9-35e1-4345-9694-2941291bd5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:44:34.059109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-ca9071b9-35e1-4345-9694-2941291bd5e7 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:44:34.060345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ca9071b9-35e1-4345-9694-2941291bd5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:44:34.154577 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 583/1171] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:44:34.162613 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13aaa31-1ee1-4aa8-a0ef-11f3b743e09b None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:44:34.163117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13aaa31-1ee1-4aa8-a0ef-11f3b743e09b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:44:34.163293 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13aaa31-1ee1-4aa8-a0ef-11f3b743e09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:44:34.239669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-f13aaa31-1ee1-4aa8-a0ef-11f3b743e09b None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:44:34.240906 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-f13aaa31-1ee1-4aa8-a0ef-11f3b743e09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:44:34.338730 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 589/1172] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:44:34.343411 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9071b9-35e1-4345-9694-2941291bd5e7 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:44:34.344638 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ca9071b9-35e1-4345-9694-2941291bd5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:44:34.356932 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9071b9-35e1-4345-9694-2941291bd5e7 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:44:34.418915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:44:34.454904 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a14c6-2a75-4224-906d-cae0309154ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:44:34.455502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a14c6-2a75-4224-906d-cae0309154ce admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:44:34.455775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a14c6-2a75-4224-906d-cae0309154ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:44:34.456043 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a14c6-2a75-4224-906d-cae0309154ce admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:44:34.456701 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a14c6-2a75-4224-906d-cae0309154ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:44:34.456998 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a14c6-2a75-4224-906d-cae0309154ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:44:34.457995 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: WARNING py.warnings [None req-b21a14c6-2a75-4224-906d-cae0309154ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:44:34.457995 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: warnings.warn(msg) Oct 30 00:44:34.457995 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: Oct 30 00:44:34.458878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a14c6-2a75-4224-906d-cae0309154ce admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:44:34.470235 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 584/1173] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:44:34.480336 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13aaa31-1ee1-4aa8-a0ef-11f3b743e09b None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:44:34.489521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13aaa31-1ee1-4aa8-a0ef-11f3b743e09b None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:44:34.552953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:44:34.586631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17905171-9631-4727-b373-df3a85c4f0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:44:34.588026 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17905171-9631-4727-b373-df3a85c4f0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:44:34.588026 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17905171-9631-4727-b373-df3a85c4f0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:44:34.588312 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17905171-9631-4727-b373-df3a85c4f0a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:44:34.588926 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17905171-9631-4727-b373-df3a85c4f0a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:44:34.589944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: WARNING py.warnings [None req-17905171-9631-4727-b373-df3a85c4f0a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:44:34.589944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: warnings.warn(msg) Oct 30 00:44:34.589944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: Oct 30 00:44:34.591134 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17905171-9631-4727-b373-df3a85c4f0a1 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:44:34.646236 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 590/1174] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:44:34 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:44:34.651011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a14c6-2a75-4224-906d-cae0309154ce admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:44:34.653005 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-b21a14c6-2a75-4224-906d-cae0309154ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:44:34.686405 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a14c6-2a75-4224-906d-cae0309154ce admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:44:34.779896 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5ba82-6f63-48af-ac26-39b8c065e75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:44:34.780541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5ba82-6f63-48af-ac26-39b8c065e75b admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:44:34.780814 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5ba82-6f63-48af-ac26-39b8c065e75b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:44:34.781082 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5ba82-6f63-48af-ac26-39b8c065e75b admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:44:34.781699 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5ba82-6f63-48af-ac26-39b8c065e75b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:44:34.782699 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: WARNING py.warnings [None req-16d5ba82-6f63-48af-ac26-39b8c065e75b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:44:34.782699 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: warnings.warn(msg) Oct 30 00:44:34.782699 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: Oct 30 00:44:34.783887 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5ba82-6f63-48af-ac26-39b8c065e75b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:44:34.828898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 585/1175] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:44:34 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:44:34.833059 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17905171-9631-4727-b373-df3a85c4f0a1 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:44:34.834552 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-17905171-9631-4727-b373-df3a85c4f0a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:44:34.896871 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17905171-9631-4727-b373-df3a85c4f0a1 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:44:34.986895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:44:34.998836 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108828cf-6a3c-49a5-9baf-286660ac4cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:44:34.999447 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108828cf-6a3c-49a5-9baf-286660ac4cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:44:34.999727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108828cf-6a3c-49a5-9baf-286660ac4cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:44:35.000002 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108828cf-6a3c-49a5-9baf-286660ac4cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:44:35.000678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108828cf-6a3c-49a5-9baf-286660ac4cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:44:35.002298 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108828cf-6a3c-49a5-9baf-286660ac4cb7 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:44:35.015081 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 591/1176] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:44:35.019104 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5ba82-6f63-48af-ac26-39b8c065e75b admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:44:35.020642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-16d5ba82-6f63-48af-ac26-39b8c065e75b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:44:35.032499 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5ba82-6f63-48af-ac26-39b8c065e75b admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:44:35.057949 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:44:35.128523 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb08fd-a1dd-48c9-b6ad-ad3e8a2198a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:44:35.129130 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb08fd-a1dd-48c9-b6ad-ad3e8a2198a7 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:44:35.129408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb08fd-a1dd-48c9-b6ad-ad3e8a2198a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:44:35.129678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb08fd-a1dd-48c9-b6ad-ad3e8a2198a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:44:35.130313 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb08fd-a1dd-48c9-b6ad-ad3e8a2198a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:44:35.131908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb08fd-a1dd-48c9-b6ad-ad3e8a2198a7 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:44:35.145044 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 586/1177] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:44:35.149260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108828cf-6a3c-49a5-9baf-286660ac4cb7 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:44:35.151357 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-108828cf-6a3c-49a5-9baf-286660ac4cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:44:35.184599 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108828cf-6a3c-49a5-9baf-286660ac4cb7 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:44:35.282583 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbf882-66dc-4a63-8e66-76d24c5a2c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:44:35.283156 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbf882-66dc-4a63-8e66-76d24c5a2c8c admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:44:35.283436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbf882-66dc-4a63-8e66-76d24c5a2c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:44:35.283710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbf882-66dc-4a63-8e66-76d24c5a2c8c admin admin] PATH_INFO: `/v3/projects/407e79b6955b48d9904324845f0bf0a0/users/2a3949e0ac94416a98a3026077201230/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:44:35.305841 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbf882-66dc-4a63-8e66-76d24c5a2c8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=407e79b6955b48d9904324845f0bf0a0, user_id=2a3949e0ac94416a98a3026077201230, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:44:35.309150 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbf882-66dc-4a63-8e66-76d24c5a2c8c admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:44:35.336053 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 592/1178] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:44:35 2022] PUT /identity/v3/projects/407e79b6955b48d9904324845f0bf0a0/users/2a3949e0ac94416a98a3026077201230/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:44:35.347373 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc2269-a4d2-4a8b-99cd-2d6dbea4c8b5 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:44:35.347659 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc2269-a4d2-4a8b-99cd-2d6dbea4c8b5 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:44:35.347931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc2269-a4d2-4a8b-99cd-2d6dbea4c8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:44:35.404440 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-a1bc2269-a4d2-4a8b-99cd-2d6dbea4c8b5 None None] MFA Rules not processed for user `2a3949e0ac94416a98a3026077201230`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:44:35.405642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a1bc2269-a4d2-4a8b-99cd-2d6dbea4c8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:44:35.483761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 587/1179] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:44:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:44:35.497645 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ffac5-98b1-42d0-a399-4b39a3cd1340 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:44:35.497931 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ffac5-98b1-42d0-a399-4b39a3cd1340 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:44:35.498205 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ffac5-98b1-42d0-a399-4b39a3cd1340 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:44:35.499564 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 593/1180] 10.222.0.36 () {54 vars in 963 bytes} [Sun Oct 30 00:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:44:35.504229 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdd136-c0e1-4642-9476-86ba65e27034 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:44:35.504520 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdd136-c0e1-4642-9476-86ba65e27034 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:44:35.504799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdd136-c0e1-4642-9476-86ba65e27034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:44:35.537269 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:44:35.585625 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-74fdd136-c0e1-4642-9476-86ba65e27034 None None] MFA Rules not processed for user `2dbe6b3bdfb3405589b8ce5aebebb93e`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:44:35.586885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-74fdd136-c0e1-4642-9476-86ba65e27034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:44:35.664976 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 588/1181] 10.222.0.36 () {58 vars in 1060 bytes} [Sun Oct 30 00:44:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:44:35.673435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ffac5-98b1-42d0-a399-4b39a3cd1340 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:44:35.674962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-428ffac5-98b1-42d0-a399-4b39a3cd1340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:44:35.727973 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:44:35.782726 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ffac5-98b1-42d0-a399-4b39a3cd1340 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:44:35.858310 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623622a-a446-4ddd-932e-270a981f627d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:44:35.858831 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623622a-a446-4ddd-932e-270a981f627d service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:44:35.859137 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623622a-a446-4ddd-932e-270a981f627d service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:44:35.859407 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623622a-a446-4ddd-932e-270a981f627d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:44:35.861018 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-1623622a-a446-4ddd-932e-270a981f627d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:44:35.962510 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623622a-a446-4ddd-932e-270a981f627d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:44:35.964591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623622a-a446-4ddd-932e-270a981f627d service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:44:35.968117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:44:35.982808 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 594/1182] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:44:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:44:36.028484 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf570e5-c00e-4002-b2cb-abfb514a3fd1 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:44:36.028918 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf570e5-c00e-4002-b2cb-abfb514a3fd1 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:44:36.029021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf570e5-c00e-4002-b2cb-abfb514a3fd1 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:44:36.030262 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 589/1183] 10.222.0.36 () {52 vars in 830 bytes} [Sun Oct 30 00:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:44:36.033795 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623622a-a446-4ddd-932e-270a981f627d service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:44:36.035015 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-1623622a-a446-4ddd-932e-270a981f627d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:44:36.133149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623622a-a446-4ddd-932e-270a981f627d service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:44:36.225216 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a2a8a-1702-494b-abfc-142ca65409e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:44:36.225860 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a2a8a-1702-494b-abfc-142ca65409e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:44:36.226129 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a2a8a-1702-494b-abfc-142ca65409e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:44:36.226381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a2a8a-1702-494b-abfc-142ca65409e1 admin admin] PATH_INFO: `/v3/users/2a3949e0ac94416a98a3026077201230` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:44:36.239265 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a2a8a-1702-494b-abfc-142ca65409e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a3949e0ac94416a98a3026077201230)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:44:36.240350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: WARNING py.warnings [None req-326a2a8a-1702-494b-abfc-142ca65409e1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:44:36.240350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: warnings.warn(msg) Oct 30 00:44:36.240350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: Oct 30 00:44:36.241562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a2a8a-1702-494b-abfc-142ca65409e1 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:44:36.254748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:44:36.311594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-326a2a8a-1702-494b-abfc-142ca65409e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a3949e0ac94416a98a3026077201230'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:44:36.312063 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-326a2a8a-1702-494b-abfc-142ca65409e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a3949e0ac94416a98a3026077201230'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:44:36.336430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-326a2a8a-1702-494b-abfc-142ca65409e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a3949e0ac94416a98a3026077201230'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:44:36.350930 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-326a2a8a-1702-494b-abfc-142ca65409e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a3949e0ac94416a98a3026077201230'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:44:36.351497 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-326a2a8a-1702-494b-abfc-142ca65409e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a3949e0ac94416a98a3026077201230'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:44:36.366211 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 595/1184] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:44:36 2022] DELETE /identity/v3/users/2a3949e0ac94416a98a3026077201230 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:44:36.371670 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc367e-3616-43ac-af31-b1fbc115b94b None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:44:36.371956 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc367e-3616-43ac-af31-b1fbc115b94b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:44:36.372288 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc367e-3616-43ac-af31-b1fbc115b94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:44:36.430020 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-45dc367e-3616-43ac-af31-b1fbc115b94b None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:44:36.431227 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-45dc367e-3616-43ac-af31-b1fbc115b94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:44:36.507395 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:44:36.520549 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 590/1185] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:44:36.531131 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a2a8a-1702-494b-abfc-142ca65409e1 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:44:36.532418 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-326a2a8a-1702-494b-abfc-142ca65409e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:44:36.633278 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a2a8a-1702-494b-abfc-142ca65409e1 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:44:36.687683 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:44:36.708338 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e304a0-56be-4a5f-b0c9-78b12ecaaf54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:44:36.708957 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e304a0-56be-4a5f-b0c9-78b12ecaaf54 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:44:36.709237 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e304a0-56be-4a5f-b0c9-78b12ecaaf54 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:44:36.709536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e304a0-56be-4a5f-b0c9-78b12ecaaf54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:44:36.711134 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e6e304a0-56be-4a5f-b0c9-78b12ecaaf54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:44:36.819733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e304a0-56be-4a5f-b0c9-78b12ecaaf54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:44:36.821495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e304a0-56be-4a5f-b0c9-78b12ecaaf54 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:44:36.840162 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 596/1186] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:44:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:44:37.165567 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc367e-3616-43ac-af31-b1fbc115b94b None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:44:37.167462 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-45dc367e-3616-43ac-af31-b1fbc115b94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:44:37.178817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc367e-3616-43ac-af31-b1fbc115b94b None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:44:37.291985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33974868-1f28-4981-b6d1-b41193f78129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:44:37.292572 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33974868-1f28-4981-b6d1-b41193f78129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:44:37.292850 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33974868-1f28-4981-b6d1-b41193f78129 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:44:37.293124 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33974868-1f28-4981-b6d1-b41193f78129 admin admin] PATH_INFO: `/v3/projects/407e79b6955b48d9904324845f0bf0a0` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:44:37.304238 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33974868-1f28-4981-b6d1-b41193f78129 admin admin] RBAC: Authorizing `identity:delete_project(project_id=407e79b6955b48d9904324845f0bf0a0)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:44:37.305290 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: WARNING py.warnings [None req-33974868-1f28-4981-b6d1-b41193f78129 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:44:37.305290 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: warnings.warn(msg) Oct 30 00:44:37.305290 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: Oct 30 00:44:37.307412 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33974868-1f28-4981-b6d1-b41193f78129 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:44:37.371888 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-33974868-1f28-4981-b6d1-b41193f78129 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '407e79b6955b48d9904324845f0bf0a0', 'request_id': 'req-33974868-1f28-4981-b6d1-b41193f78129'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:44:37.380963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-33974868-1f28-4981-b6d1-b41193f78129 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '407e79b6955b48d9904324845f0bf0a0', 'request_id': 'req-33974868-1f28-4981-b6d1-b41193f78129'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:44:37.388376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-33974868-1f28-4981-b6d1-b41193f78129 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '407e79b6955b48d9904324845f0bf0a0', 'request_id': 'req-33974868-1f28-4981-b6d1-b41193f78129'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:44:37.419584 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-33974868-1f28-4981-b6d1-b41193f78129 admin admin] The token cache is being invalidate because project 407e79b6955b48d9904324845f0bf0a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:44:37.419871 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-33974868-1f28-4981-b6d1-b41193f78129 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:44:37.420421 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-33974868-1f28-4981-b6d1-b41193f78129 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:44:37.421715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 591/1187] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:44:37 2022] DELETE /identity/v3/projects/407e79b6955b48d9904324845f0bf0a0 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:44:46.837264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=62 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:44:47.417586 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:46:09.278850 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8908f-3049-465b-9759-c11faa5913ab None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:09.279461 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8908f-3049-465b-9759-c11faa5913ab None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:09.279461 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8908f-3049-465b-9759-c11faa5913ab None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:09.280823 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 597/1188] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 00:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:46:09.285686 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7374b-185b-4327-a5c8-cac636a8562d None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:09.286149 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7374b-185b-4327-a5c8-cac636a8562d None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:09.286255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7374b-185b-4327-a5c8-cac636a8562d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:09.367808 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-07b7374b-185b-4327-a5c8-cac636a8562d None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:46:09.369074 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-07b7374b-185b-4327-a5c8-cac636a8562d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:46:09.472187 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 592/1189] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 00:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:46:09.628869 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8908f-3049-465b-9759-c11faa5913ab None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:46:09.630992 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-24c8908f-3049-465b-9759-c11faa5913ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:46:09.750226 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8908f-3049-465b-9759-c11faa5913ab None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:46:09.755630 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536f72e-9cc9-434b-b8c7-c14a2f5df218 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:46:09.757046 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536f72e-9cc9-434b-b8c7-c14a2f5df218 service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:09.757456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536f72e-9cc9-434b-b8c7-c14a2f5df218 service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:09.757731 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536f72e-9cc9-434b-b8c7-c14a2f5df218 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:09.765601 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536f72e-9cc9-434b-b8c7-c14a2f5df218 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:46:09.767185 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536f72e-9cc9-434b-b8c7-c14a2f5df218 service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:46:09.776916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 598/1190] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:46:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:46:10.848528 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a42fbb-89a0-4cc2-ab9c-59ec15cc9cac None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:10.849099 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a42fbb-89a0-4cc2-ab9c-59ec15cc9cac None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:10.849099 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a42fbb-89a0-4cc2-ab9c-59ec15cc9cac None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:10.851152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 593/1191] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 00:46:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:46:10.856011 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0e19d-5eb8-468d-bc03-ae450e346c4a None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:10.856446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0e19d-5eb8-468d-bc03-ae450e346c4a None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:10.856582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0e19d-5eb8-468d-bc03-ae450e346c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:10.877393 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-e0f0e19d-5eb8-468d-bc03-ae450e346c4a None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:46:10.878592 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e0f0e19d-5eb8-468d-bc03-ae450e346c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:46:10.887592 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 599/1192] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 00:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:46:11.040550 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a42fbb-89a0-4cc2-ab9c-59ec15cc9cac None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:46:11.056558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a42fbb-89a0-4cc2-ab9c-59ec15cc9cac None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:46:11.062077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4d1d8-41c1-4fe5-960d-b931f42e32b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:46:11.062689 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4d1d8-41c1-4fe5-960d-b931f42e32b0 service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:11.062974 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4d1d8-41c1-4fe5-960d-b931f42e32b0 service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:11.063250 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4d1d8-41c1-4fe5-960d-b931f42e32b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:11.072480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4d1d8-41c1-4fe5-960d-b931f42e32b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:46:11.074077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4d1d8-41c1-4fe5-960d-b931f42e32b0 service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:46:11.084452 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 594/1193] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:46:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:46:11.106319 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856aeba8-5500-4282-bd93-e76c4f907868 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:11.106627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856aeba8-5500-4282-bd93-e76c4f907868 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:11.106921 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856aeba8-5500-4282-bd93-e76c4f907868 None None] PATH_INFO: `/v3` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:11.108483 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 600/1194] 10.222.0.20 () {54 vars in 925 bytes} [Sun Oct 30 00:46:11 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:46:11.115196 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e02243-7811-4803-a09c-bbb6fba3f77a None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:11.116217 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e02243-7811-4803-a09c-bbb6fba3f77a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:11.116515 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e02243-7811-4803-a09c-bbb6fba3f77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:11.197217 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-d7e02243-7811-4803-a09c-bbb6fba3f77a None None] MFA Rules not processed for user `5ded89d48d6c4a15a5c5829f594f8776`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:46:11.198470 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d7e02243-7811-4803-a09c-bbb6fba3f77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:46:11.217491 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:46:11.275446 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 595/1195] 10.222.0.20 () {58 vars in 1015 bytes} [Sun Oct 30 00:46:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:46:20.178196 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fae78-b22d-4250-b420-b25e2edafd04 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:20.178693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fae78-b22d-4250-b420-b25e2edafd04 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:20.178830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fae78-b22d-4250-b420-b25e2edafd04 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:20.180065 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 601/1196] 10.222.0.36 () {54 vars in 961 bytes} [Sun Oct 30 00:46:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:46:20.185256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d662ff-ee48-4e87-b58d-75bebf1b5d44 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:20.185670 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d662ff-ee48-4e87-b58d-75bebf1b5d44 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:20.185810 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d662ff-ee48-4e87-b58d-75bebf1b5d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:20.205587 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-d6d662ff-ee48-4e87-b58d-75bebf1b5d44 None None] MFA Rules not processed for user `5ded89d48d6c4a15a5c5829f594f8776`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:46:20.206800 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d6d662ff-ee48-4e87-b58d-75bebf1b5d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:46:20.215429 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 596/1197] 10.222.0.36 () {58 vars in 1058 bytes} [Sun Oct 30 00:46:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:46:20.223056 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489fae78-b22d-4250-b420-b25e2edafd04 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:46:20.244796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489fae78-b22d-4250-b420-b25e2edafd04 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:46:20.249858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf048cab-d4a2-4bb4-bab7-9f4774fec0c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:46:20.250458 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf048cab-d4a2-4bb4-bab7-9f4774fec0c6 service placement] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:20.250738 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf048cab-d4a2-4bb4-bab7-9f4774fec0c6 service placement] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:20.251010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf048cab-d4a2-4bb4-bab7-9f4774fec0c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:20.259141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf048cab-d4a2-4bb4-bab7-9f4774fec0c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:46:20.260754 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf048cab-d4a2-4bb4-bab7-9f4774fec0c6 service placement] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:46:20.270153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 602/1198] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:46:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:46:22.556679 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7784d2-831d-47b2-956d-1f9db8f24e03 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:22.557592 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7784d2-831d-47b2-956d-1f9db8f24e03 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:22.557592 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7784d2-831d-47b2-956d-1f9db8f24e03 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:22.559103 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 597/1199] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 00:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:46:22.564180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873c119-82bd-4be1-bcf6-4f47c05fcf39 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:22.564618 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873c119-82bd-4be1-bcf6-4f47c05fcf39 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:22.564721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873c119-82bd-4be1-bcf6-4f47c05fcf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:22.585210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-8873c119-82bd-4be1-bcf6-4f47c05fcf39 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:46:22.586412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-8873c119-82bd-4be1-bcf6-4f47c05fcf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:46:22.594297 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 603/1200] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 00:46:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:46:22.752885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7784d2-831d-47b2-956d-1f9db8f24e03 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:46:22.767792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7784d2-831d-47b2-956d-1f9db8f24e03 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:46:22.773805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0aa5a-fc20-4668-a0d8-37c80dfa64ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:46:22.774552 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0aa5a-fc20-4668-a0d8-37c80dfa64ba service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:22.774935 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0aa5a-fc20-4668-a0d8-37c80dfa64ba service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:22.775304 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0aa5a-fc20-4668-a0d8-37c80dfa64ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:22.783628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0aa5a-fc20-4668-a0d8-37c80dfa64ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:46:22.785232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0aa5a-fc20-4668-a0d8-37c80dfa64ba service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:46:22.794813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 598/1201] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:46:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:46:23.859815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd0bf1-6020-4964-8c53-2657ee6240a2 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:23.860438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd0bf1-6020-4964-8c53-2657ee6240a2 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:23.860438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd0bf1-6020-4964-8c53-2657ee6240a2 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:23.861737 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 604/1202] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 30 00:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:46:23.866524 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe8cdd-afca-4ec7-8245-322d0d3798f9 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:23.866963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe8cdd-afca-4ec7-8245-322d0d3798f9 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:23.867113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe8cdd-afca-4ec7-8245-322d0d3798f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:23.886826 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-63fe8cdd-afca-4ec7-8245-322d0d3798f9 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:46:23.888017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-63fe8cdd-afca-4ec7-8245-322d0d3798f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:46:23.896950 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 599/1203] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 30 00:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:46:24.047624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd0bf1-6020-4964-8c53-2657ee6240a2 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:46:24.062365 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd0bf1-6020-4964-8c53-2657ee6240a2 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:46:24.068587 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7166671-38ac-4587-8160-68fdd7e563a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:46:24.069456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7166671-38ac-4587-8160-68fdd7e563a9 service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:24.069735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7166671-38ac-4587-8160-68fdd7e563a9 service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:24.069999 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7166671-38ac-4587-8160-68fdd7e563a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:24.078028 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7166671-38ac-4587-8160-68fdd7e563a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:46:24.079607 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7166671-38ac-4587-8160-68fdd7e563a9 service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:46:24.089190 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 605/1204] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:46:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:46:28.258609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8fc1b-31b0-4c37-b8a8-bd8673b781b8 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:28.259195 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8fc1b-31b0-4c37-b8a8-bd8673b781b8 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:28.259195 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8fc1b-31b0-4c37-b8a8-bd8673b781b8 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:28.260530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 600/1205] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 00:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:46:28.265727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a021f85-7f57-4fe7-bc27-9b54604da148 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:28.266211 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a021f85-7f57-4fe7-bc27-9b54604da148 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:28.266342 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a021f85-7f57-4fe7-bc27-9b54604da148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:28.286234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-8a021f85-7f57-4fe7-bc27-9b54604da148 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:46:28.287430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-8a021f85-7f57-4fe7-bc27-9b54604da148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:46:28.295168 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 606/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 00:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:46:28.601546 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8fc1b-31b0-4c37-b8a8-bd8673b781b8 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:46:28.603212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-48c8fc1b-31b0-4c37-b8a8-bd8673b781b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:46:28.681326 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8fc1b-31b0-4c37-b8a8-bd8673b781b8 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:46:28.684902 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbb605-5106-4d35-9af2-bc29bbc9991f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:46:28.685505 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbb605-5106-4d35-9af2-bc29bbc9991f service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:28.685785 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbb605-5106-4d35-9af2-bc29bbc9991f service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:28.686056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbb605-5106-4d35-9af2-bc29bbc9991f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:28.695840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbb605-5106-4d35-9af2-bc29bbc9991f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:46:28.697301 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbb605-5106-4d35-9af2-bc29bbc9991f service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:46:28.706552 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 601/1207] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:46:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:46:30.906279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b14d27-57ca-4275-8985-175485862cf0 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:30.906827 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b14d27-57ca-4275-8985-175485862cf0 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:30.906827 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b14d27-57ca-4275-8985-175485862cf0 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:30.908347 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 607/1208] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 00:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:46:30.914484 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51ecc4-d479-45dd-8340-896b68c27e15 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:30.914899 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51ecc4-d479-45dd-8340-896b68c27e15 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:30.915032 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51ecc4-d479-45dd-8340-896b68c27e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:30.935218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-7c51ecc4-d479-45dd-8340-896b68c27e15 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:46:30.936470 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-7c51ecc4-d479-45dd-8340-896b68c27e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:46:30.944841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 602/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 00:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:46:31.232646 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b14d27-57ca-4275-8985-175485862cf0 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:46:31.244221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b14d27-57ca-4275-8985-175485862cf0 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:46:31.247728 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f110dd-edc6-436b-9e91-e1dc5c42a426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:46:31.248356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f110dd-edc6-436b-9e91-e1dc5c42a426 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:31.248634 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f110dd-edc6-436b-9e91-e1dc5c42a426 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:31.248904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f110dd-edc6-436b-9e91-e1dc5c42a426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:31.257796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f110dd-edc6-436b-9e91-e1dc5c42a426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:46:31.259231 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f110dd-edc6-436b-9e91-e1dc5c42a426 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:46:31.268373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 608/1210] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:46:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:46:35.168456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb997e91-6abb-45e5-93fe-252f45b40e2c None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:35.169054 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb997e91-6abb-45e5-93fe-252f45b40e2c None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:35.169054 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb997e91-6abb-45e5-93fe-252f45b40e2c None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:35.170317 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 603/1211] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 00:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:46:35.176670 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff6164-05f1-4a4d-ae49-405344e000fa None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:35.178054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff6164-05f1-4a4d-ae49-405344e000fa None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:35.178317 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff6164-05f1-4a4d-ae49-405344e000fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:35.198039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-98ff6164-05f1-4a4d-ae49-405344e000fa None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:46:35.199216 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-98ff6164-05f1-4a4d-ae49-405344e000fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:46:35.207554 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 609/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 00:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:46:35.517979 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb997e91-6abb-45e5-93fe-252f45b40e2c None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:46:35.520345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-cb997e91-6abb-45e5-93fe-252f45b40e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:46:35.585416 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:46:35.600287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb997e91-6abb-45e5-93fe-252f45b40e2c None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:46:35.603848 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ede82-343b-42a4-b8de-9e7783effcb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:46:35.604497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ede82-343b-42a4-b8de-9e7783effcb5 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:35.604778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ede82-343b-42a4-b8de-9e7783effcb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:35.605050 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ede82-343b-42a4-b8de-9e7783effcb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:35.614117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ede82-343b-42a4-b8de-9e7783effcb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:46:35.615536 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ede82-343b-42a4-b8de-9e7783effcb5 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:46:35.625235 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 604/1213] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:46:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:46:37.821010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd40f0e-8595-41e3-bd00-352ee68c9405 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:37.821573 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd40f0e-8595-41e3-bd00-352ee68c9405 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:37.821573 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd40f0e-8595-41e3-bd00-352ee68c9405 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:37.822888 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 610/1214] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 30 00:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:46:37.828177 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af27b30-4ed9-4066-8f0d-d96868bd47e6 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:37.828575 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af27b30-4ed9-4066-8f0d-d96868bd47e6 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:37.828733 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af27b30-4ed9-4066-8f0d-d96868bd47e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:37.849825 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-5af27b30-4ed9-4066-8f0d-d96868bd47e6 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:46:37.851019 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-5af27b30-4ed9-4066-8f0d-d96868bd47e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:46:37.858804 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 605/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 30 00:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:46:38.165898 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd40f0e-8595-41e3-bd00-352ee68c9405 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:46:38.167560 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-7fd40f0e-8595-41e3-bd00-352ee68c9405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:46:38.175074 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=84 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:46:38.182337 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd40f0e-8595-41e3-bd00-352ee68c9405 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:46:38.185886 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc014351-6a39-4313-a1b7-2843c4cb9e70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:46:38.186486 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc014351-6a39-4313-a1b7-2843c4cb9e70 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:46:38.186767 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc014351-6a39-4313-a1b7-2843c4cb9e70 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:46:38.187040 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc014351-6a39-4313-a1b7-2843c4cb9e70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:46:38.196233 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc014351-6a39-4313-a1b7-2843c4cb9e70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:46:38.197693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc014351-6a39-4313-a1b7-2843c4cb9e70 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:46:38.207097 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 611/1216] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:46:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:46:47.844192 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=11 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:46:48.202907 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=5 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:03.328065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b97718-e7a6-4f47-84b8-f28602493096 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:03.328736 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b97718-e7a6-4f47-84b8-f28602493096 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:03.328736 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b97718-e7a6-4f47-84b8-f28602493096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:03.355463 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-20b97718-e7a6-4f47-84b8-f28602493096 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:03.356753 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-20b97718-e7a6-4f47-84b8-f28602493096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:03.368563 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 606/1217] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:48:03.379264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23467f6-d1c0-41e4-bf01-4ea9a23e46eb None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:03.380431 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23467f6-d1c0-41e4-bf01-4ea9a23e46eb None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:03.380431 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23467f6-d1c0-41e4-bf01-4ea9a23e46eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:03.417361 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-a23467f6-d1c0-41e4-bf01-4ea9a23e46eb None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:03.418602 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a23467f6-d1c0-41e4-bf01-4ea9a23e46eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:03.428069 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 612/1218] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:03.432665 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b97718-e7a6-4f47-84b8-f28602493096 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:03.448966 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b97718-e7a6-4f47-84b8-f28602493096 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:03.457561 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488a579-7e3f-4c99-9d97-e865f5a5a617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:03.458192 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488a579-7e3f-4c99-9d97-e865f5a5a617 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:03.458472 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488a579-7e3f-4c99-9d97-e865f5a5a617 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:03.458747 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488a579-7e3f-4c99-9d97-e865f5a5a617 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:03.459400 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488a579-7e3f-4c99-9d97-e865f5a5a617 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:03.459701 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488a579-7e3f-4c99-9d97-e865f5a5a617 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:03.461347 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488a579-7e3f-4c99-9d97-e865f5a5a617 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:03.482318 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 607/1219] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:03.491107 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23467f6-d1c0-41e4-bf01-4ea9a23e46eb None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:03.506666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23467f6-d1c0-41e4-bf01-4ea9a23e46eb None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:03.512792 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dec2d-5422-470a-80f3-c27595bf9b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:03.513433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dec2d-5422-470a-80f3-c27595bf9b4d admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:03.513716 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dec2d-5422-470a-80f3-c27595bf9b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:03.513990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dec2d-5422-470a-80f3-c27595bf9b4d admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:03.514817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1dec2d-5422-470a-80f3-c27595bf9b4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:03.517358 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1dec2d-5422-470a-80f3-c27595bf9b4d admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:03.585688 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 613/1220] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:03 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:03.589991 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488a579-7e3f-4c99-9d97-e865f5a5a617 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:03.607270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488a579-7e3f-4c99-9d97-e865f5a5a617 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:03.613904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de6f35-4831-4e93-a745-c852a8b720cc None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:03.614260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de6f35-4831-4e93-a745-c852a8b720cc None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:03.614568 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de6f35-4831-4e93-a745-c852a8b720cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:03.646687 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-40de6f35-4831-4e93-a745-c852a8b720cc None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:03.647923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-40de6f35-4831-4e93-a745-c852a8b720cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:03.710436 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4c48e-1965-4936-90b9-84abae0a2e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:03.711068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4c48e-1965-4936-90b9-84abae0a2e05 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:03.711352 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4c48e-1965-4936-90b9-84abae0a2e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:03.711905 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4c48e-1965-4936-90b9-84abae0a2e05 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:03.712601 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4c48e-1965-4936-90b9-84abae0a2e05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:03.714592 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4c48e-1965-4936-90b9-84abae0a2e05 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:03.764361 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 608/1221] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:03 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:03.766197 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a0c87-64ee-4c33-b0c7-d8439312201e None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:03.766526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a0c87-64ee-4c33-b0c7-d8439312201e None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:03.766810 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a0c87-64ee-4c33-b0c7-d8439312201e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:03.782979 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 614/1222] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:03.783901 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de6f35-4831-4e93-a745-c852a8b720cc None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:03.799354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de6f35-4831-4e93-a745-c852a8b720cc None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:03.801546 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-df8a0c87-64ee-4c33-b0c7-d8439312201e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:03.802814 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-df8a0c87-64ee-4c33-b0c7-d8439312201e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:03.808976 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29135e20-2768-4089-a65c-fc749658adea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:03.809627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29135e20-2768-4089-a65c-fc749658adea admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:03.809912 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29135e20-2768-4089-a65c-fc749658adea admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:03.810192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29135e20-2768-4089-a65c-fc749658adea admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:03.810853 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29135e20-2768-4089-a65c-fc749658adea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:03.817911 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29135e20-2768-4089-a65c-fc749658adea admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:03.823056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 609/1223] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:03.824467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3036c-0bc7-4e9d-9c77-9e318f717dc8 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:03.825160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3036c-0bc7-4e9d-9c77-9e318f717dc8 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:03.825160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3036c-0bc7-4e9d-9c77-9e318f717dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:03.831481 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 615/1224] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:03.842621 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095cd25-0fad-462d-8666-e863da912327 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:03.843507 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095cd25-0fad-462d-8666-e863da912327 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:03.843748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095cd25-0fad-462d-8666-e863da912327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:03.856846 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-14c3036c-0bc7-4e9d-9c77-9e318f717dc8 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:03.858072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-14c3036c-0bc7-4e9d-9c77-9e318f717dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:03.883643 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 610/1225] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:03.884528 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3036c-0bc7-4e9d-9c77-9e318f717dc8 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:03.893254 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-c095cd25-0fad-462d-8666-e863da912327 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:03.894472 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c095cd25-0fad-462d-8666-e863da912327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:03.901665 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3036c-0bc7-4e9d-9c77-9e318f717dc8 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:03.913630 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f0e79-b297-4914-a783-b4bfd7d2ddd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:03.914661 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f0e79-b297-4914-a783-b4bfd7d2ddd0 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:03.915152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f0e79-b297-4914-a783-b4bfd7d2ddd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:03.915691 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f0e79-b297-4914-a783-b4bfd7d2ddd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:03.916617 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f0e79-b297-4914-a783-b4bfd7d2ddd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:03.918599 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f0e79-b297-4914-a783-b4bfd7d2ddd0 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:03.926235 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 616/1226] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:48:03.926235 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836edd46-7f28-413c-982d-1375870a5c81 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:03.926235 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836edd46-7f28-413c-982d-1375870a5c81 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:03.926235 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836edd46-7f28-413c-982d-1375870a5c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:03.940665 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 611/1227] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:03.942726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f0e79-b297-4914-a783-b4bfd7d2ddd0 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:03.966341 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-836edd46-7f28-413c-982d-1375870a5c81 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:03.967670 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-836edd46-7f28-413c-982d-1375870a5c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:03.978223 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f0e79-b297-4914-a783-b4bfd7d2ddd0 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:03.987711 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37997003-e76d-4c9d-aba3-d9df3d17ee6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:03.988372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37997003-e76d-4c9d-aba3-d9df3d17ee6a admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:03.988661 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37997003-e76d-4c9d-aba3-d9df3d17ee6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:03.988937 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37997003-e76d-4c9d-aba3-d9df3d17ee6a admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:03.994121 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 617/1228] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:03.994800 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836edd46-7f28-413c-982d-1375870a5c81 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:03.998052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37997003-e76d-4c9d-aba3-d9df3d17ee6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:03.998384 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37997003-e76d-4c9d-aba3-d9df3d17ee6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.000149 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37997003-e76d-4c9d-aba3-d9df3d17ee6a admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.008302 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836edd46-7f28-413c-982d-1375870a5c81 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.014959 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f0ea3-3328-4991-a422-95dd30fbc45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.015926 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f0ea3-3328-4991-a422-95dd30fbc45e admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.016481 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f0ea3-3328-4991-a422-95dd30fbc45e admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.016764 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 612/1229] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:04.017368 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f0ea3-3328-4991-a422-95dd30fbc45e admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.018252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f0ea3-3328-4991-a422-95dd30fbc45e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:04.018574 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafb2d2-79d3-4994-86dc-89c1130aef43 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.018913 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafb2d2-79d3-4994-86dc-89c1130aef43 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.019190 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafb2d2-79d3-4994-86dc-89c1130aef43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.019541 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f0ea3-3328-4991-a422-95dd30fbc45e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.021048 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: WARNING py.warnings [None req-304f0ea3-3328-4991-a422-95dd30fbc45e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:48:04.021048 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: warnings.warn(msg) Oct 30 00:48:04.021048 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: Oct 30 00:48:04.022419 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f0ea3-3328-4991-a422-95dd30fbc45e admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.037629 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 618/1230] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:04.038890 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f0ea3-3328-4991-a422-95dd30fbc45e admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.041379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-faafb2d2-79d3-4994-86dc-89c1130aef43 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:04.042629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-faafb2d2-79d3-4994-86dc-89c1130aef43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:04.050918 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f0ea3-3328-4991-a422-95dd30fbc45e admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.052944 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 613/1231] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:04.054644 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c918ff7-405b-46c8-af92-ea4d60fa18b5 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.054983 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c918ff7-405b-46c8-af92-ea4d60fa18b5 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.055264 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c918ff7-405b-46c8-af92-ea4d60fa18b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.057442 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca632933-db1b-4af0-a4f1-f3b1b9de6179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.058365 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca632933-db1b-4af0-a4f1-f3b1b9de6179 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.058864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca632933-db1b-4af0-a4f1-f3b1b9de6179 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.059366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca632933-db1b-4af0-a4f1-f3b1b9de6179 admin admin] PATH_INFO: `/v3/projects/7d10f71e30d04db39566e439f4fba6a1/users/220cbd96393049598539054b1d2a180e/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.077244 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca632933-db1b-4af0-a4f1-f3b1b9de6179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d10f71e30d04db39566e439f4fba6a1, user_id=220cbd96393049598539054b1d2a180e, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.078350 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-0c918ff7-405b-46c8-af92-ea4d60fa18b5 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:04.079980 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-0c918ff7-405b-46c8-af92-ea4d60fa18b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:04.082111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca632933-db1b-4af0-a4f1-f3b1b9de6179 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.094677 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 614/1232] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:04.095878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076f6b4-deaa-4623-ba05-ae40a792fafd None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.096207 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076f6b4-deaa-4623-ba05-ae40a792fafd None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.096482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076f6b4-deaa-4623-ba05-ae40a792fafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.102304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 619/1233] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:04 2022] PUT /identity/v3/projects/7d10f71e30d04db39566e439f4fba6a1/users/220cbd96393049598539054b1d2a180e/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:04.103674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca632933-db1b-4af0-a4f1-f3b1b9de6179 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.125388 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-9076f6b4-deaa-4623-ba05-ae40a792fafd None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:04.126742 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-9076f6b4-deaa-4623-ba05-ae40a792fafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:04.169775 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca632933-db1b-4af0-a4f1-f3b1b9de6179 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.175296 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a17f3d-deb2-49bf-a1b2-8555df49d40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.175907 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a17f3d-deb2-49bf-a1b2-8555df49d40f admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.176288 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a17f3d-deb2-49bf-a1b2-8555df49d40f admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.176704 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a17f3d-deb2-49bf-a1b2-8555df49d40f admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.177339 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a17f3d-deb2-49bf-a1b2-8555df49d40f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.179261 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a17f3d-deb2-49bf-a1b2-8555df49d40f admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.184657 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:04.189656 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 615/1234] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:04.190664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076f6b4-deaa-4623-ba05-ae40a792fafd None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.201139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076f6b4-deaa-4623-ba05-ae40a792fafd None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.206497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ffae9-4ddc-41b2-80f5-f26bae4455c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.207298 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ffae9-4ddc-41b2-80f5-f26bae4455c2 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.207575 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ffae9-4ddc-41b2-80f5-f26bae4455c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.207848 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ffae9-4ddc-41b2-80f5-f26bae4455c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.208150 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 620/1235] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/projects => generated 405 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:04.208512 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ffae9-4ddc-41b2-80f5-f26bae4455c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.209535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: WARNING py.warnings [None req-4d1ffae9-4ddc-41b2-80f5-f26bae4455c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:48:04.209535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: warnings.warn(msg) Oct 30 00:48:04.209535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: Oct 30 00:48:04.210727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ffae9-4ddc-41b2-80f5-f26bae4455c2 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.213233 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d2510-fe43-44e5-aa2d-276771968860 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.213528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d2510-fe43-44e5-aa2d-276771968860 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.213813 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d2510-fe43-44e5-aa2d-276771968860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.234843 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-0e8d2510-fe43-44e5-aa2d-276771968860 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:04.235101 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:04.235932 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-0e8d2510-fe43-44e5-aa2d-276771968860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:04.242466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 616/1236] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:04.244031 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631e3e6-777c-43ba-906c-9e965f9d4667 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.244354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631e3e6-777c-43ba-906c-9e965f9d4667 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.244617 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631e3e6-777c-43ba-906c-9e965f9d4667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.263905 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-4631e3e6-777c-43ba-906c-9e965f9d4667 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:04.266181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4631e3e6-777c-43ba-906c-9e965f9d4667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:04.298381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 621/1237] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:04.299643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b2654-11b8-4ce5-a089-987ec66f4108 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.299935 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b2654-11b8-4ce5-a089-987ec66f4108 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.300247 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b2654-11b8-4ce5-a089-987ec66f4108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.320197 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-1c7b2654-11b8-4ce5-a089-987ec66f4108 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:04.321390 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-1c7b2654-11b8-4ce5-a089-987ec66f4108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:04.332636 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 617/1238] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:04.333977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060eda1-e549-4b10-82ad-66aad70e9375 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.334271 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060eda1-e549-4b10-82ad-66aad70e9375 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.334546 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060eda1-e549-4b10-82ad-66aad70e9375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.363280 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-8060eda1-e549-4b10-82ad-66aad70e9375 None None] MFA Rules not processed for user `220cbd96393049598539054b1d2a180e`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:04.364516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-8060eda1-e549-4b10-82ad-66aad70e9375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:04.384811 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 622/1239] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:04.386075 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03654dbb-2a35-4dbb-8723-1cc9d9a82177 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.386361 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03654dbb-2a35-4dbb-8723-1cc9d9a82177 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.386637 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03654dbb-2a35-4dbb-8723-1cc9d9a82177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.406749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-03654dbb-2a35-4dbb-8723-1cc9d9a82177 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:04.407947 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-03654dbb-2a35-4dbb-8723-1cc9d9a82177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:04.415803 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 623/1240] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:04.416613 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03654dbb-2a35-4dbb-8723-1cc9d9a82177 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.428993 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03654dbb-2a35-4dbb-8723-1cc9d9a82177 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.433520 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a37337-2b6c-4aa6-9e32-e70fb103f7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.434108 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a37337-2b6c-4aa6-9e32-e70fb103f7de admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.434383 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a37337-2b6c-4aa6-9e32-e70fb103f7de admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.434492 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 618/1241] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:04.434707 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a37337-2b6c-4aa6-9e32-e70fb103f7de admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.435373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a37337-2b6c-4aa6-9e32-e70fb103f7de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:04.435680 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a37337-2b6c-4aa6-9e32-e70fb103f7de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.437335 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a37337-2b6c-4aa6-9e32-e70fb103f7de admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.440388 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060eda1-e549-4b10-82ad-66aad70e9375 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.449844 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 624/1242] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:04.451045 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060eda1-e549-4b10-82ad-66aad70e9375 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.457020 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a37337-2b6c-4aa6-9e32-e70fb103f7de admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.457945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b99ba6-e419-4177-b5d9-40e79629a8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.458562 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b99ba6-e419-4177-b5d9-40e79629a8d7 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.458841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b99ba6-e419-4177-b5d9-40e79629a8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.459115 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b99ba6-e419-4177-b5d9-40e79629a8d7 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.459760 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b99ba6-e419-4177-b5d9-40e79629a8d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.461874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b99ba6-e419-4177-b5d9-40e79629a8d7 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.470623 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a37337-2b6c-4aa6-9e32-e70fb103f7de admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.475827 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6098be-4ccd-4059-98bb-d6634d330880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.476537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6098be-4ccd-4059-98bb-d6634d330880 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.476883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6098be-4ccd-4059-98bb-d6634d330880 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.477217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6098be-4ccd-4059-98bb-d6634d330880 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.477963 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6098be-4ccd-4059-98bb-d6634d330880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.479051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: WARNING py.warnings [None req-1f6098be-4ccd-4059-98bb-d6634d330880 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:48:04.479051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: warnings.warn(msg) Oct 30 00:48:04.479051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: Oct 30 00:48:04.480351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6098be-4ccd-4059-98bb-d6634d330880 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.490693 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 619/1243] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/users => generated 303 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:04.491865 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b99ba6-e419-4177-b5d9-40e79629a8d7 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.503548 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 625/1244] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/users => generated 294 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:04.504622 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b99ba6-e419-4177-b5d9-40e79629a8d7 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.504817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f7a46-8f6c-44c8-a3e0-5f6cb4a68760 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.505097 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f7a46-8f6c-44c8-a3e0-5f6cb4a68760 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.505367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f7a46-8f6c-44c8-a3e0-5f6cb4a68760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.510155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cadfe6-5410-48b2-a285-9dcb2034464a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.510752 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cadfe6-5410-48b2-a285-9dcb2034464a admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.511029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cadfe6-5410-48b2-a285-9dcb2034464a admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.511301 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cadfe6-5410-48b2-a285-9dcb2034464a admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.511947 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cadfe6-5410-48b2-a285-9dcb2034464a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:04.512555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cadfe6-5410-48b2-a285-9dcb2034464a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.514256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cadfe6-5410-48b2-a285-9dcb2034464a admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.524805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-9d5f7a46-8f6c-44c8-a3e0-5f6cb4a68760 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:04.526000 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-9d5f7a46-8f6c-44c8-a3e0-5f6cb4a68760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:04.526582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 620/1245] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:04.527517 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cadfe6-5410-48b2-a285-9dcb2034464a admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.536304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 626/1246] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:04.537524 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bed46f-4642-4a1c-a939-353f67f3cb60 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.537813 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bed46f-4642-4a1c-a939-353f67f3cb60 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.538084 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bed46f-4642-4a1c-a939-353f67f3cb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.538951 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cadfe6-5410-48b2-a285-9dcb2034464a admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.543762 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c44c59-f382-4759-a095-353e488084f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.544387 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c44c59-f382-4759-a095-353e488084f2 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.544668 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c44c59-f382-4759-a095-353e488084f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.544939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c44c59-f382-4759-a095-353e488084f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.545575 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c44c59-f382-4759-a095-353e488084f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:04.545876 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c44c59-f382-4759-a095-353e488084f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.548032 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c44c59-f382-4759-a095-353e488084f2 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.558122 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-20bed46f-4642-4a1c-a939-353f67f3cb60 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:04.559436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-20bed46f-4642-4a1c-a939-353f67f3cb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:04.559552 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 621/1247] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:04.560895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9201b-4de9-4ebd-b930-4a201390686e None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.561184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9201b-4de9-4ebd-b930-4a201390686e None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.561463 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9201b-4de9-4ebd-b930-4a201390686e None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.563390 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 622/1248] 10.222.0.36 () {54 vars in 961 bytes} [Sun Oct 30 00:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:04.563975 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9201b-4de9-4ebd-b930-4a201390686e None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.574374 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 627/1249] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:04.575270 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bed46f-4642-4a1c-a939-353f67f3cb60 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.579826 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9201b-4de9-4ebd-b930-4a201390686e None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.584252 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd61272-4636-4312-8630-df4d0dfced8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.584839 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd61272-4636-4312-8630-df4d0dfced8e admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.585078 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bed46f-4642-4a1c-a939-353f67f3cb60 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.585193 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd61272-4636-4312-8630-df4d0dfced8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.585368 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd61272-4636-4312-8630-df4d0dfced8e admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.586035 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd61272-4636-4312-8630-df4d0dfced8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.587920 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd61272-4636-4312-8630-df4d0dfced8e admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.589653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc84a29-8759-41e4-94f1-f7877a7ad4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.590241 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc84a29-8759-41e4-94f1-f7877a7ad4a5 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.590525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc84a29-8759-41e4-94f1-f7877a7ad4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.590801 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc84a29-8759-41e4-94f1-f7877a7ad4a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.591432 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc84a29-8759-41e4-94f1-f7877a7ad4a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.593159 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc84a29-8759-41e4-94f1-f7877a7ad4a5 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.603104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 628/1250] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:04.603897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc84a29-8759-41e4-94f1-f7877a7ad4a5 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.615104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc84a29-8759-41e4-94f1-f7877a7ad4a5 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.622825 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23802a-7572-4ef6-b0a3-a3f6d1d88c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.623006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 623/1251] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:04.623113 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23802a-7572-4ef6-b0a3-a3f6d1d88c51 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.623204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd61272-4636-4312-8630-df4d0dfced8e admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.623303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23802a-7572-4ef6-b0a3-a3f6d1d88c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.623303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23802a-7572-4ef6-b0a3-a3f6d1d88c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.623303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23802a-7572-4ef6-b0a3-a3f6d1d88c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.623303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23802a-7572-4ef6-b0a3-a3f6d1d88c51 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.636710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 629/1252] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:04.636710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23802a-7572-4ef6-b0a3-a3f6d1d88c51 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.644575 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23802a-7572-4ef6-b0a3-a3f6d1d88c51 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.646413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:04.657878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:04.685720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd61272-4636-4312-8630-df4d0dfced8e admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.695807 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da10766-6a82-47b4-99b9-f71959b58c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.696458 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da10766-6a82-47b4-99b9-f71959b58c72 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.696739 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da10766-6a82-47b4-99b9-f71959b58c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.697011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da10766-6a82-47b4-99b9-f71959b58c72 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.697648 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da10766-6a82-47b4-99b9-f71959b58c72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.699555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da10766-6a82-47b4-99b9-f71959b58c72 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.707699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4b7be-7060-40f0-88be-a4c7a8ba2ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.708345 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4b7be-7060-40f0-88be-a4c7a8ba2ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.708631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4b7be-7060-40f0-88be-a4c7a8ba2ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.708910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4b7be-7060-40f0-88be-a4c7a8ba2ae5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.709585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d4b7be-7060-40f0-88be-a4c7a8ba2ae5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:04.709893 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d4b7be-7060-40f0-88be-a4c7a8ba2ae5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.711663 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d4b7be-7060-40f0-88be-a4c7a8ba2ae5 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.731451 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 630/1253] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:04.731451 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4b7be-7060-40f0-88be-a4c7a8ba2ae5 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.731777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 624/1254] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/projects => generated 391 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:04.731777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4d00f-fa6a-42d1-8d4a-7d0176daabd7 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.731777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4d00f-fa6a-42d1-8d4a-7d0176daabd7 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.731777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4d00f-fa6a-42d1-8d4a-7d0176daabd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.735581 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4b7be-7060-40f0-88be-a4c7a8ba2ae5 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.769211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-4ea4d00f-fa6a-42d1-8d4a-7d0176daabd7 None None] MFA Rules not processed for user `adbba7be39a64230bab160ce77ce7586`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:04.770422 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4ea4d00f-fa6a-42d1-8d4a-7d0176daabd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:04.794448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499c2dc-847e-4722-93e2-6e04a68d061b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.795060 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499c2dc-847e-4722-93e2-6e04a68d061b admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.795335 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499c2dc-847e-4722-93e2-6e04a68d061b admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.795604 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499c2dc-847e-4722-93e2-6e04a68d061b admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.796451 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499c2dc-847e-4722-93e2-6e04a68d061b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.798449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499c2dc-847e-4722-93e2-6e04a68d061b admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.825411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 631/1255] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:04.826357 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499c2dc-847e-4722-93e2-6e04a68d061b admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.844604 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 625/1256] 10.222.0.36 () {58 vars in 1058 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:04.845633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4d00f-fa6a-42d1-8d4a-7d0176daabd7 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.855948 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4d00f-fa6a-42d1-8d4a-7d0176daabd7 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.897455 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499c2dc-847e-4722-93e2-6e04a68d061b admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.902446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68422784-75a5-4c71-ada5-767f4754e892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.903057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68422784-75a5-4c71-ada5-767f4754e892 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.903344 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68422784-75a5-4c71-ada5-767f4754e892 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.903619 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68422784-75a5-4c71-ada5-767f4754e892 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.904312 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68422784-75a5-4c71-ada5-767f4754e892 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:04.904619 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68422784-75a5-4c71-ada5-767f4754e892 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.907003 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68422784-75a5-4c71-ada5-767f4754e892 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.915156 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb5ecb-4253-41e2-9a3f-5e351521872a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.915773 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb5ecb-4253-41e2-9a3f-5e351521872a admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.916052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb5ecb-4253-41e2-9a3f-5e351521872a admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.916358 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb5ecb-4253-41e2-9a3f-5e351521872a admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.917023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb5ecb-4253-41e2-9a3f-5e351521872a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.923650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 632/1257] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:04.923861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb5ecb-4253-41e2-9a3f-5e351521872a admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.923974 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68422784-75a5-4c71-ada5-767f4754e892 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.929208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 626/1258] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:04.930152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb5ecb-4253-41e2-9a3f-5e351521872a admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.938089 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68422784-75a5-4c71-ada5-767f4754e892 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.941121 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb5ecb-4253-41e2-9a3f-5e351521872a admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.943027 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83b1ac-df7d-4874-914a-dbda704cf736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.943686 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83b1ac-df7d-4874-914a-dbda704cf736 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.943972 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83b1ac-df7d-4874-914a-dbda704cf736 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.944287 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83b1ac-df7d-4874-914a-dbda704cf736 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.944949 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83b1ac-df7d-4874-914a-dbda704cf736 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.945731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766382ab-5621-4197-8046-97b5d04a8611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.946321 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766382ab-5621-4197-8046-97b5d04a8611 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.946601 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766382ab-5621-4197-8046-97b5d04a8611 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.946871 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766382ab-5621-4197-8046-97b5d04a8611 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.946973 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83b1ac-df7d-4874-914a-dbda704cf736 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.947496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766382ab-5621-4197-8046-97b5d04a8611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.949126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766382ab-5621-4197-8046-97b5d04a8611 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.959441 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 627/1259] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:04.960345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766382ab-5621-4197-8046-97b5d04a8611 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.971809 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766382ab-5621-4197-8046-97b5d04a8611 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.976387 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 633/1260] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/users => generated 290 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:04.976387 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83b1ac-df7d-4874-914a-dbda704cf736 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:04.979647 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1983bd-580c-4b88-b7fd-f40077a869b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.980272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1983bd-580c-4b88-b7fd-f40077a869b4 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.980552 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1983bd-580c-4b88-b7fd-f40077a869b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.980827 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1983bd-580c-4b88-b7fd-f40077a869b4 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.981448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1983bd-580c-4b88-b7fd-f40077a869b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.983243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83b1ac-df7d-4874-914a-dbda704cf736 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:04.983495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1983bd-580c-4b88-b7fd-f40077a869b4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.988048 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdb9d1-ced1-4586-aa1b-f0f329b85886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:04.988655 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdb9d1-ced1-4586-aa1b-f0f329b85886 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:04.988934 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdb9d1-ced1-4586-aa1b-f0f329b85886 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:04.989203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdb9d1-ced1-4586-aa1b-f0f329b85886 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:04.989815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fdb9d1-ced1-4586-aa1b-f0f329b85886 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:04.991719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fdb9d1-ced1-4586-aa1b-f0f329b85886 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:04.993858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:05.005138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:05.007065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 628/1261] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/users => generated 296 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:05.008107 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1983bd-580c-4b88-b7fd-f40077a869b4 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.019830 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1983bd-580c-4b88-b7fd-f40077a869b4 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.027537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f10e4-ed12-43a0-91ec-fe38e504c57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.028522 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f10e4-ed12-43a0-91ec-fe38e504c57e admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.028959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f10e4-ed12-43a0-91ec-fe38e504c57e admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.029489 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f10e4-ed12-43a0-91ec-fe38e504c57e admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.030371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f10e4-ed12-43a0-91ec-fe38e504c57e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.032683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f10e4-ed12-43a0-91ec-fe38e504c57e admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.034367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 634/1262] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:04 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:05.035262 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdb9d1-ced1-4586-aa1b-f0f329b85886 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.058203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 629/1263] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:05 2022] POST /identity/v3/users => generated 284 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:05.059386 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f10e4-ed12-43a0-91ec-fe38e504c57e admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.070099 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f10e4-ed12-43a0-91ec-fe38e504c57e admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.108022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdb9d1-ced1-4586-aa1b-f0f329b85886 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.114210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bbf31-24ef-45c0-8896-7f353254117d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.114823 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bbf31-24ef-45c0-8896-7f353254117d service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.115106 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bbf31-24ef-45c0-8896-7f353254117d service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.115411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bbf31-24ef-45c0-8896-7f353254117d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.130310 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3ac7e-1c5c-48dd-a4a4-82356084fb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.130861 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3ac7e-1c5c-48dd-a4a4-82356084fb5d admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.131139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3ac7e-1c5c-48dd-a4a4-82356084fb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.131421 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3ac7e-1c5c-48dd-a4a4-82356084fb5d admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.132048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3ac7e-1c5c-48dd-a4a4-82356084fb5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.134147 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3ac7e-1c5c-48dd-a4a4-82356084fb5d admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.162319 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 630/1264] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:05 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:05.165352 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3ac7e-1c5c-48dd-a4a4-82356084fb5d admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.175314 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293bbf31-24ef-45c0-8896-7f353254117d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.177034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293bbf31-24ef-45c0-8896-7f353254117d service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.186082 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 635/1265] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:05.187574 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bbf31-24ef-45c0-8896-7f353254117d service nova] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.197791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bbf31-24ef-45c0-8896-7f353254117d service nova] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.227510 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3ac7e-1c5c-48dd-a4a4-82356084fb5d admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.232251 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88def4e-0b34-4748-8213-d2eeee21facf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.232919 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88def4e-0b34-4748-8213-d2eeee21facf admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.233193 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88def4e-0b34-4748-8213-d2eeee21facf admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.233999 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88def4e-0b34-4748-8213-d2eeee21facf admin admin] PATH_INFO: `/v3/projects/39791c410e0e495e94e692fda6bd5e23/users/1557eb6485d345d0b7b54d4b32b1d77b/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.248634 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88def4e-0b34-4748-8213-d2eeee21facf admin admin] RBAC: Authorizing `identity:create_grant(project_id=39791c410e0e495e94e692fda6bd5e23, user_id=1557eb6485d345d0b7b54d4b32b1d77b, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.252324 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88def4e-0b34-4748-8213-d2eeee21facf admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.267051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56110e-ef0b-4bb7-94b4-d1a6a29e108d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.267613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 631/1266] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:05 2022] PUT /identity/v3/projects/39791c410e0e495e94e692fda6bd5e23/users/1557eb6485d345d0b7b54d4b32b1d77b/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:05.268543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56110e-ef0b-4bb7-94b4-d1a6a29e108d admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.268667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88def4e-0b34-4748-8213-d2eeee21facf admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.269253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56110e-ef0b-4bb7-94b4-d1a6a29e108d admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.269773 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56110e-ef0b-4bb7-94b4-d1a6a29e108d admin admin] PATH_INFO: `/v3/projects/0f3b2aeb0e094f539e4affd66ea66272/users/df1ee15eb3bc468188a9d8c1af46529c/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.288404 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56110e-ef0b-4bb7-94b4-d1a6a29e108d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f3b2aeb0e094f539e4affd66ea66272, user_id=df1ee15eb3bc468188a9d8c1af46529c, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.291100 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56110e-ef0b-4bb7-94b4-d1a6a29e108d admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.305857 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 636/1267] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:05 2022] PUT /identity/v3/projects/0f3b2aeb0e094f539e4affd66ea66272/users/df1ee15eb3bc468188a9d8c1af46529c/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:05.311127 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56110e-ef0b-4bb7-94b4-d1a6a29e108d admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.322175 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:05.324648 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:05.344731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88def4e-0b34-4748-8213-d2eeee21facf admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.385340 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56110e-ef0b-4bb7-94b4-d1a6a29e108d admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.392413 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1c536-4315-42b7-91f1-767c3bbb53d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.393580 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1c536-4315-42b7-91f1-767c3bbb53d6 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.394069 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1c536-4315-42b7-91f1-767c3bbb53d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.394448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1c536-4315-42b7-91f1-767c3bbb53d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.395345 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1c536-4315-42b7-91f1-767c3bbb53d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.397837 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1c536-4315-42b7-91f1-767c3bbb53d6 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.404151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd68bb-7ab2-43fe-8813-f51771217d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.404877 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd68bb-7ab2-43fe-8813-f51771217d67 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.405776 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd68bb-7ab2-43fe-8813-f51771217d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.406185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd68bb-7ab2-43fe-8813-f51771217d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.406976 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd68bb-7ab2-43fe-8813-f51771217d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.408724 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd68bb-7ab2-43fe-8813-f51771217d67 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.410734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 637/1268] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:05.411355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1c536-4315-42b7-91f1-767c3bbb53d6 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.423107 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 632/1269] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:05.424052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd68bb-7ab2-43fe-8813-f51771217d67 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.427203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1c536-4315-42b7-91f1-767c3bbb53d6 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.432443 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792b678-3aa8-4f3b-87dc-f51ffbf9218f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.433050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792b678-3aa8-4f3b-87dc-f51ffbf9218f admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.433330 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792b678-3aa8-4f3b-87dc-f51ffbf9218f admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.433600 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792b678-3aa8-4f3b-87dc-f51ffbf9218f admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.434085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd68bb-7ab2-43fe-8813-f51771217d67 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.434222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792b678-3aa8-4f3b-87dc-f51ffbf9218f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.436196 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792b678-3aa8-4f3b-87dc-f51ffbf9218f admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.438780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16b17c-2601-4cd1-a183-fd307b68c300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.439682 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16b17c-2601-4cd1-a183-fd307b68c300 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.440020 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16b17c-2601-4cd1-a183-fd307b68c300 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.440384 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16b17c-2601-4cd1-a183-fd307b68c300 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.441104 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16b17c-2601-4cd1-a183-fd307b68c300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.442738 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16b17c-2601-4cd1-a183-fd307b68c300 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.453177 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 633/1270] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:05.454047 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16b17c-2601-4cd1-a183-fd307b68c300 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.459778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 638/1271] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:05 2022] POST /identity/v3/users => generated 288 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:05.461047 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571810f-fced-4da6-8e26-b5cf1f724ced None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.461329 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571810f-fced-4da6-8e26-b5cf1f724ced None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.461613 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571810f-fced-4da6-8e26-b5cf1f724ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.466478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16b17c-2601-4cd1-a183-fd307b68c300 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.472764 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2375b7-a5dd-413a-8527-775d0633c36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.473377 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2375b7-a5dd-413a-8527-775d0633c36c admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.473664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2375b7-a5dd-413a-8527-775d0633c36c admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.473967 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2375b7-a5dd-413a-8527-775d0633c36c admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.474602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2375b7-a5dd-413a-8527-775d0633c36c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.476551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2375b7-a5dd-413a-8527-775d0633c36c admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.491119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-8571810f-fced-4da6-8e26-b5cf1f724ced None None] MFA Rules not processed for user `1557eb6485d345d0b7b54d4b32b1d77b`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:05.492375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-8571810f-fced-4da6-8e26-b5cf1f724ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:05.500235 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 634/1272] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:05 2022] POST /identity/v3/users => generated 294 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:05.501598 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5af71-9106-47fa-adbd-522cebe0faf1 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.501903 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5af71-9106-47fa-adbd-522cebe0faf1 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.502162 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5af71-9106-47fa-adbd-522cebe0faf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.521591 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-73e5af71-9106-47fa-adbd-522cebe0faf1 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:05.522819 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-73e5af71-9106-47fa-adbd-522cebe0faf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:05.531866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 635/1273] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:05.532687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5af71-9106-47fa-adbd-522cebe0faf1 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.543041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5af71-9106-47fa-adbd-522cebe0faf1 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.548367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523ba58-a90c-4d68-95c4-be322a37c6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.548966 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523ba58-a90c-4d68-95c4-be322a37c6bd admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.549248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523ba58-a90c-4d68-95c4-be322a37c6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.549524 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523ba58-a90c-4d68-95c4-be322a37c6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.550165 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f523ba58-a90c-4d68-95c4-be322a37c6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.551778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f523ba58-a90c-4d68-95c4-be322a37c6bd admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.561868 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 636/1274] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:05.562660 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523ba58-a90c-4d68-95c4-be322a37c6bd admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.566063 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 639/1275] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:05 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:05.566063 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f1e1f-2451-4e42-a00f-3b6660af9c3e None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.566063 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f1e1f-2451-4e42-a00f-3b6660af9c3e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.566063 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f1e1f-2451-4e42-a00f-3b6660af9c3e None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.567176 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 640/1276] 10.222.0.36 () {54 vars in 912 bytes} [Sun Oct 30 00:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:05.567920 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f1e1f-2451-4e42-a00f-3b6660af9c3e None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.577719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523ba58-a90c-4d68-95c4-be322a37c6bd admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.582175 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f1e1f-2451-4e42-a00f-3b6660af9c3e None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.582492 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc4060-c18a-405c-841a-0bc152afe57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.583098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc4060-c18a-405c-841a-0bc152afe57d admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.583387 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc4060-c18a-405c-841a-0bc152afe57d admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.583663 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc4060-c18a-405c-841a-0bc152afe57d admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.584333 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc4060-c18a-405c-841a-0bc152afe57d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.585983 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc4060-c18a-405c-841a-0bc152afe57d admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.587034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c59885-578b-4b98-898e-642f43ea8985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.587860 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c59885-578b-4b98-898e-642f43ea8985 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.588543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c59885-578b-4b98-898e-642f43ea8985 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.588932 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c59885-578b-4b98-898e-642f43ea8985 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.589571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c59885-578b-4b98-898e-642f43ea8985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.591649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c59885-578b-4b98-898e-642f43ea8985 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.597066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 637/1277] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:05.597956 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc4060-c18a-405c-841a-0bc152afe57d admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.606181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 641/1278] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:05.606181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c59885-578b-4b98-898e-642f43ea8985 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.611119 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc4060-c18a-405c-841a-0bc152afe57d admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.613900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c59885-578b-4b98-898e-642f43ea8985 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.615696 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f95e41-f6e6-4841-b46a-77a4eb154e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.616448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f95e41-f6e6-4841-b46a-77a4eb154e29 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.616729 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f95e41-f6e6-4841-b46a-77a4eb154e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.616997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f95e41-f6e6-4841-b46a-77a4eb154e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.617621 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f95e41-f6e6-4841-b46a-77a4eb154e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.618665 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c647ed-aa2e-4677-93dc-105adf699414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.619193 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f95e41-f6e6-4841-b46a-77a4eb154e29 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.619293 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c647ed-aa2e-4677-93dc-105adf699414 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.619542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c647ed-aa2e-4677-93dc-105adf699414 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.619815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c647ed-aa2e-4677-93dc-105adf699414 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.620486 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c647ed-aa2e-4677-93dc-105adf699414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.622065 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c647ed-aa2e-4677-93dc-105adf699414 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.630755 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 638/1279] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:05.631383 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f95e41-f6e6-4841-b46a-77a4eb154e29 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.635576 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 642/1280] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:05.635576 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c647ed-aa2e-4677-93dc-105adf699414 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.645852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c647ed-aa2e-4677-93dc-105adf699414 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.650460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77ad11-ae32-4b23-b0b1-3ed21547492c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.651093 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77ad11-ae32-4b23-b0b1-3ed21547492c admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.651369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77ad11-ae32-4b23-b0b1-3ed21547492c admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.651640 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77ad11-ae32-4b23-b0b1-3ed21547492c admin admin] PATH_INFO: `/v3/projects/999d27163d544dff836ab7d90355f0d6/users/350f5e4238f448949b2c372aabbfc835/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.666076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77ad11-ae32-4b23-b0b1-3ed21547492c admin admin] RBAC: Authorizing `identity:create_grant(project_id=999d27163d544dff836ab7d90355f0d6, user_id=350f5e4238f448949b2c372aabbfc835, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.668762 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77ad11-ae32-4b23-b0b1-3ed21547492c admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.683133 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 643/1281] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:05 2022] PUT /identity/v3/projects/999d27163d544dff836ab7d90355f0d6/users/350f5e4238f448949b2c372aabbfc835/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:05.684157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77ad11-ae32-4b23-b0b1-3ed21547492c admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.693380 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f95e41-f6e6-4841-b46a-77a4eb154e29 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.719049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:05.729095 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:05.747934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dea77c-35a0-402f-8cb7-5cdf7c163e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.748585 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dea77c-35a0-402f-8cb7-5cdf7c163e50 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.748862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dea77c-35a0-402f-8cb7-5cdf7c163e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.749767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dea77c-35a0-402f-8cb7-5cdf7c163e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.750290 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77ad11-ae32-4b23-b0b1-3ed21547492c admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.754703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1eb26-a45e-4d7e-a70d-b0e1b2ee3267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.755315 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1eb26-a45e-4d7e-a70d-b0e1b2ee3267 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.755596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1eb26-a45e-4d7e-a70d-b0e1b2ee3267 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.755883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1eb26-a45e-4d7e-a70d-b0e1b2ee3267 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.756540 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1eb26-a45e-4d7e-a70d-b0e1b2ee3267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.758451 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1eb26-a45e-4d7e-a70d-b0e1b2ee3267 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.758929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dea77c-35a0-402f-8cb7-5cdf7c163e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.760427 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dea77c-35a0-402f-8cb7-5cdf7c163e50 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.770992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 639/1282] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:05.772261 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872d67d-d843-492b-930f-677d4c467809 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.772552 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872d67d-d843-492b-930f-677d4c467809 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.772883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872d67d-d843-492b-930f-677d4c467809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.786969 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 644/1283] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:05 2022] POST /identity/v3/projects => generated 405 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:05.787988 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1eb26-a45e-4d7e-a70d-b0e1b2ee3267 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.793852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-8872d67d-d843-492b-930f-677d4c467809 None None] MFA Rules not processed for user `adbba7be39a64230bab160ce77ce7586`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:05.795231 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-8872d67d-d843-492b-930f-677d4c467809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:05.863848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1eb26-a45e-4d7e-a70d-b0e1b2ee3267 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.869538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ca536-dc42-4aa1-a7a3-816f14cc5182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.870317 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ca536-dc42-4aa1-a7a3-816f14cc5182 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.870668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ca536-dc42-4aa1-a7a3-816f14cc5182 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.871022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ca536-dc42-4aa1-a7a3-816f14cc5182 admin admin] PATH_INFO: `/v3/projects/e1b5b4b97dfd468f923996968764d206/users/8135391c507d4f06aeb7e74160bbd038/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.878725 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 640/1284] 10.222.0.36 () {58 vars in 1009 bytes} [Sun Oct 30 00:48:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:05.879741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872d67d-d843-492b-930f-677d4c467809 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.891821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ca536-dc42-4aa1-a7a3-816f14cc5182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1b5b4b97dfd468f923996968764d206, user_id=8135391c507d4f06aeb7e74160bbd038, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.896674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ca536-dc42-4aa1-a7a3-816f14cc5182 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.897575 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872d67d-d843-492b-930f-677d4c467809 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:05.902947 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26192360-b1ef-41ee-81c0-5d952e1eba82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:05.903649 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26192360-b1ef-41ee-81c0-5d952e1eba82 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:05.904166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26192360-b1ef-41ee-81c0-5d952e1eba82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:05.904467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26192360-b1ef-41ee-81c0-5d952e1eba82 admin admin] PATH_INFO: `/v3/projects/4fecc8a40c97490ebb78edb763c2e4d4/users/258b8f46bd88425e89da0d81cbdb681b/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:05.916969 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 645/1285] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:05 2022] PUT /identity/v3/projects/e1b5b4b97dfd468f923996968764d206/users/8135391c507d4f06aeb7e74160bbd038/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:05.921459 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ca536-dc42-4aa1-a7a3-816f14cc5182 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:05.923494 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26192360-b1ef-41ee-81c0-5d952e1eba82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fecc8a40c97490ebb78edb763c2e4d4, user_id=258b8f46bd88425e89da0d81cbdb681b, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:05.926889 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26192360-b1ef-41ee-81c0-5d952e1eba82 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:05.948575 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 641/1286] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:05 2022] PUT /identity/v3/projects/4fecc8a40c97490ebb78edb763c2e4d4/users/258b8f46bd88425e89da0d81cbdb681b/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:05.949824 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26192360-b1ef-41ee-81c0-5d952e1eba82 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:06.016509 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ca536-dc42-4aa1-a7a3-816f14cc5182 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:06.042514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26192360-b1ef-41ee-81c0-5d952e1eba82 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:06.049122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e9fa7-2740-485d-8b79-ec7941136346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:06.049896 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e9fa7-2740-485d-8b79-ec7941136346 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:06.050240 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e9fa7-2740-485d-8b79-ec7941136346 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:06.050535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e9fa7-2740-485d-8b79-ec7941136346 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:06.051805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e9fa7-2740-485d-8b79-ec7941136346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:06.053866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e9fa7-2740-485d-8b79-ec7941136346 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:06.073184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 642/1287] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:06.073310 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:06.073409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066649cf-94ea-4fda-b1c3-4d5cc8c30276 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:06.073409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066649cf-94ea-4fda-b1c3-4d5cc8c30276 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:06.073409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066649cf-94ea-4fda-b1c3-4d5cc8c30276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:06.086123 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cdf752-dbc5-47ba-9db3-cd1696a7d162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:06.086887 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdf752-dbc5-47ba-9db3-cd1696a7d162 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:06.087264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdf752-dbc5-47ba-9db3-cd1696a7d162 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:06.087650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cdf752-dbc5-47ba-9db3-cd1696a7d162 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:06.088419 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cdf752-dbc5-47ba-9db3-cd1696a7d162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:06.090128 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cdf752-dbc5-47ba-9db3-cd1696a7d162 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:06.094601 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-066649cf-94ea-4fda-b1c3-4d5cc8c30276 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:06.095870 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-066649cf-94ea-4fda-b1c3-4d5cc8c30276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:06.102816 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 646/1288] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:06.103833 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 643/1289] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:06.104122 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cdf752-dbc5-47ba-9db3-cd1696a7d162 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:06.104676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066649cf-94ea-4fda-b1c3-4d5cc8c30276 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:06.116918 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cdf752-dbc5-47ba-9db3-cd1696a7d162 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:06.121908 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9d2fa-1033-4aaf-b5a4-538bcdd913d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:06.122636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9d2fa-1033-4aaf-b5a4-538bcdd913d7 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:06.123061 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9d2fa-1033-4aaf-b5a4-538bcdd913d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:06.123430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9d2fa-1033-4aaf-b5a4-538bcdd913d7 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:06.124489 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a9d2fa-1033-4aaf-b5a4-538bcdd913d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:06.126444 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a9d2fa-1033-4aaf-b5a4-538bcdd913d7 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:06.151295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 647/1290] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:06 2022] POST /identity/v3/users => generated 309 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:06.153368 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168bbf3-ea19-4fc3-9118-1f38c0cc4396 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:06.153834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168bbf3-ea19-4fc3-9118-1f38c0cc4396 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:06.154510 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168bbf3-ea19-4fc3-9118-1f38c0cc4396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:06.155035 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:06.171758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066649cf-94ea-4fda-b1c3-4d5cc8c30276 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:06.175964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd17af-42ec-4fcc-a11b-775279c49db7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:06.176746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd17af-42ec-4fcc-a11b-775279c49db7 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:06.177039 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd17af-42ec-4fcc-a11b-775279c49db7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:06.177318 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd17af-42ec-4fcc-a11b-775279c49db7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:06.178767 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-a168bbf3-ea19-4fc3-9118-1f38c0cc4396 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:06.179978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a168bbf3-ea19-4fc3-9118-1f38c0cc4396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:06.198054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 648/1291] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:06.200567 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d6344-7ef8-4ab2-9299-da927cc30fc0 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:06.201011 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d6344-7ef8-4ab2-9299-da927cc30fc0 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:06.201656 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d6344-7ef8-4ab2-9299-da927cc30fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:06.245029 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-506d6344-7ef8-4ab2-9299-da927cc30fc0 None None] MFA Rules not processed for user `258b8f46bd88425e89da0d81cbdb681b`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:06.246766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-506d6344-7ef8-4ab2-9299-da927cc30fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:06.289373 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd17af-42ec-4fcc-a11b-775279c49db7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:06.291402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd17af-42ec-4fcc-a11b-775279c49db7 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:06.305268 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 644/1292] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:06.306115 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd17af-42ec-4fcc-a11b-775279c49db7 service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:06.319331 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd17af-42ec-4fcc-a11b-775279c49db7 service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:06.325435 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b2cbe-6a3e-472e-95dc-8bbfef97464b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:06.326172 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b2cbe-6a3e-472e-95dc-8bbfef97464b admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:06.326455 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b2cbe-6a3e-472e-95dc-8bbfef97464b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:06.326767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b2cbe-6a3e-472e-95dc-8bbfef97464b admin admin] PATH_INFO: `/v3/projects/3a9494126a7b463c801ff049f0241b09/users/6290ef6f826d40c7a975be6eae0f1ec5/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:06.337584 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 649/1293] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:06.342218 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d6344-7ef8-4ab2-9299-da927cc30fc0 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:06.353916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d6344-7ef8-4ab2-9299-da927cc30fc0 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:06.355809 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647b2cbe-6a3e-472e-95dc-8bbfef97464b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a9494126a7b463c801ff049f0241b09, user_id=6290ef6f826d40c7a975be6eae0f1ec5, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:06.358642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647b2cbe-6a3e-472e-95dc-8bbfef97464b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:06.359026 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6e8e1-091c-44cf-af4a-9313b69422d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:06.359850 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6e8e1-091c-44cf-af4a-9313b69422d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:06.360153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6e8e1-091c-44cf-af4a-9313b69422d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:06.360437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6e8e1-091c-44cf-af4a-9313b69422d1 admin admin] PATH_INFO: `/v3/projects/bb5d551adcf64bb8aea88a6c142cea5e/users/5ac3fdb86ac24fa493706f84e1aa40a8/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:06.377323 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 645/1294] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:06 2022] PUT /identity/v3/projects/3a9494126a7b463c801ff049f0241b09/users/6290ef6f826d40c7a975be6eae0f1ec5/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:06.378285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b2cbe-6a3e-472e-95dc-8bbfef97464b admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:06.379607 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6e8e1-091c-44cf-af4a-9313b69422d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb5d551adcf64bb8aea88a6c142cea5e, user_id=5ac3fdb86ac24fa493706f84e1aa40a8, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:06.380027 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-647b2cbe-6a3e-472e-95dc-8bbfef97464b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:06.382461 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6e8e1-091c-44cf-af4a-9313b69422d1 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:06.401243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 650/1295] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:06 2022] PUT /identity/v3/projects/bb5d551adcf64bb8aea88a6c142cea5e/users/5ac3fdb86ac24fa493706f84e1aa40a8/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:06.402194 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6e8e1-091c-44cf-af4a-9313b69422d1 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:06.453276 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b2cbe-6a3e-472e-95dc-8bbfef97464b admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:06.483460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6e8e1-091c-44cf-af4a-9313b69422d1 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:06.488921 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359cea7-a542-4c84-9eb6-4fac98c0f3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:06.489760 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359cea7-a542-4c84-9eb6-4fac98c0f3de admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:06.490208 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359cea7-a542-4c84-9eb6-4fac98c0f3de admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:06.490638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359cea7-a542-4c84-9eb6-4fac98c0f3de admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:06.491442 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359cea7-a542-4c84-9eb6-4fac98c0f3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:06.493447 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359cea7-a542-4c84-9eb6-4fac98c0f3de admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:06.506192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 651/1296] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:06.507282 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359cea7-a542-4c84-9eb6-4fac98c0f3de admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:06.527049 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080963d-7ab9-4838-85cd-a1b30f7dd0d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:06.527780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080963d-7ab9-4838-85cd-a1b30f7dd0d0 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:06.528183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080963d-7ab9-4838-85cd-a1b30f7dd0d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:06.528938 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080963d-7ab9-4838-85cd-a1b30f7dd0d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:06.542930 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080963d-7ab9-4838-85cd-a1b30f7dd0d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:06.545596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080963d-7ab9-4838-85cd-a1b30f7dd0d0 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:06.548256 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:06.570722 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 646/1297] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:06.571906 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080963d-7ab9-4838-85cd-a1b30f7dd0d0 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:06.591916 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:06.652156 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080963d-7ab9-4838-85cd-a1b30f7dd0d0 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:06.657257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549868b-83b5-421b-9765-5f049b0be0fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:06.657631 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549868b-83b5-421b-9765-5f049b0be0fa service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:06.658282 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549868b-83b5-421b-9765-5f049b0be0fa service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:06.658700 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549868b-83b5-421b-9765-5f049b0be0fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:06.676292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359cea7-a542-4c84-9eb6-4fac98c0f3de admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:06.687404 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c99990-a73e-40b3-93f9-7a2e297352aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:06.691233 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c99990-a73e-40b3-93f9-7a2e297352aa service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:06.693144 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c99990-a73e-40b3-93f9-7a2e297352aa service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:06.694669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c99990-a73e-40b3-93f9-7a2e297352aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:06.723144 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c99990-a73e-40b3-93f9-7a2e297352aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:06.730196 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c99990-a73e-40b3-93f9-7a2e297352aa service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:06.745358 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6549868b-83b5-421b-9765-5f049b0be0fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:06.747339 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6549868b-83b5-421b-9765-5f049b0be0fa service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:06.757583 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 652/1298] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:06.759136 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad1986-5f7b-480b-a4fa-4ed80ba83b68 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:06.761658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 647/1299] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:06.771947 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9d0e2-e63f-42ef-bf17-25260c8b3e72 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:06.772278 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9d0e2-e63f-42ef-bf17-25260c8b3e72 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:06.772611 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9d0e2-e63f-42ef-bf17-25260c8b3e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:06.773896 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad1986-5f7b-480b-a4fa-4ed80ba83b68 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:06.774378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad1986-5f7b-480b-a4fa-4ed80ba83b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:06.797323 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-d8e9d0e2-e63f-42ef-bf17-25260c8b3e72 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:06.798707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d8e9d0e2-e63f-42ef-bf17-25260c8b3e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:06.818212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 648/1300] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:06.822103 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9d0e2-e63f-42ef-bf17-25260c8b3e72 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:06.840893 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9d0e2-e63f-42ef-bf17-25260c8b3e72 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:06.844125 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-64ad1986-5f7b-480b-a4fa-4ed80ba83b68 None None] MFA Rules not processed for user `6290ef6f826d40c7a975be6eae0f1ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:06.849723 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-64ad1986-5f7b-480b-a4fa-4ed80ba83b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:06.856633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db417e0f-2df5-4a10-8453-4d356b46f371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:06.857359 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db417e0f-2df5-4a10-8453-4d356b46f371 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:06.857640 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db417e0f-2df5-4a10-8453-4d356b46f371 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:06.857931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db417e0f-2df5-4a10-8453-4d356b46f371 admin admin] PATH_INFO: `/v3/projects/86f31b7a33f04f9d8b20794bfca91f10/users/f03c979471d64e1d9638a9c4eef1f8e1/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:06.904127 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db417e0f-2df5-4a10-8453-4d356b46f371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86f31b7a33f04f9d8b20794bfca91f10, user_id=f03c979471d64e1d9638a9c4eef1f8e1, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:06.908021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db417e0f-2df5-4a10-8453-4d356b46f371 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:06.943049 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 649/1301] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:06 2022] PUT /identity/v3/projects/86f31b7a33f04f9d8b20794bfca91f10/users/f03c979471d64e1d9638a9c4eef1f8e1/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:06.948062 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db417e0f-2df5-4a10-8453-4d356b46f371 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:07.039111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 653/1302] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:07.041050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad1986-5f7b-480b-a4fa-4ed80ba83b68 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:07.064072 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad1986-5f7b-480b-a4fa-4ed80ba83b68 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:07.174335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db417e0f-2df5-4a10-8453-4d356b46f371 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:07.195741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312b07a-cd6c-4393-a29e-29b74ccb8863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:07.199432 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312b07a-cd6c-4393-a29e-29b74ccb8863 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:07.199783 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312b07a-cd6c-4393-a29e-29b74ccb8863 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:07.199962 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312b07a-cd6c-4393-a29e-29b74ccb8863 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:07.205732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d8f8f-0b9e-4d0a-9163-82d7fefc6138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:07.206377 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d8f8f-0b9e-4d0a-9163-82d7fefc6138 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:07.206663 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d8f8f-0b9e-4d0a-9163-82d7fefc6138 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:07.206828 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a312b07a-cd6c-4393-a29e-29b74ccb8863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:07.206927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d8f8f-0b9e-4d0a-9163-82d7fefc6138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:07.213330 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a312b07a-cd6c-4393-a29e-29b74ccb8863 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:07.235032 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d8f8f-0b9e-4d0a-9163-82d7fefc6138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:07.239636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d8f8f-0b9e-4d0a-9163-82d7fefc6138 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:07.258077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 650/1303] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:07.262417 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312b07a-cd6c-4393-a29e-29b74ccb8863 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:07.277303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 654/1304] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:07.278160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d8f8f-0b9e-4d0a-9163-82d7fefc6138 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:07.303873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312b07a-cd6c-4393-a29e-29b74ccb8863 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:07.310047 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d8f8f-0b9e-4d0a-9163-82d7fefc6138 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:07.326521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026faed5-171e-47dd-bfe8-06f6f75016be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:07.327059 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026faed5-171e-47dd-bfe8-06f6f75016be admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:07.327354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026faed5-171e-47dd-bfe8-06f6f75016be admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:07.327626 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026faed5-171e-47dd-bfe8-06f6f75016be admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:07.328316 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026faed5-171e-47dd-bfe8-06f6f75016be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:07.330226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026faed5-171e-47dd-bfe8-06f6f75016be admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:07.334314 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f11caf-d685-4076-9c08-3715c188c77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:07.335061 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f11caf-d685-4076-9c08-3715c188c77b admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:07.335407 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f11caf-d685-4076-9c08-3715c188c77b admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:07.335696 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f11caf-d685-4076-9c08-3715c188c77b admin admin] PATH_INFO: `/v3/projects/86f31b7a33f04f9d8b20794bfca91f10/users/f03c979471d64e1d9638a9c4eef1f8e1/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:07.338260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f11caf-d685-4076-9c08-3715c188c77b admin admin] RBAC: Authorizing `identity:create_grant(project_id=86f31b7a33f04f9d8b20794bfca91f10, user_id=f03c979471d64e1d9638a9c4eef1f8e1, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:07.346153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f11caf-d685-4076-9c08-3715c188c77b admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:07.374801 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 655/1305] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:07 2022] PUT /identity/v3/projects/86f31b7a33f04f9d8b20794bfca91f10/users/f03c979471d64e1d9638a9c4eef1f8e1/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:07.385439 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d04644-7cc4-460c-9b6a-61eea71ff187 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:07.385737 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d04644-7cc4-460c-9b6a-61eea71ff187 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:07.386018 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d04644-7cc4-460c-9b6a-61eea71ff187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:07.426395 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 651/1306] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:07 2022] POST /identity/v3/projects => generated 387 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:07.433808 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026faed5-171e-47dd-bfe8-06f6f75016be admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:07.461647 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:07.461935 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-86d04644-7cc4-460c-9b6a-61eea71ff187 None None] MFA Rules not processed for user `f03c979471d64e1d9638a9c4eef1f8e1`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:07.463252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-86d04644-7cc4-460c-9b6a-61eea71ff187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:07.550270 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:07.663529 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 656/1307] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:07 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:07.675509 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d04644-7cc4-460c-9b6a-61eea71ff187 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:07.692601 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026faed5-171e-47dd-bfe8-06f6f75016be admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:07.710908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf36fd-9742-4474-97b9-cd5abb641d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:07.711554 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf36fd-9742-4474-97b9-cd5abb641d40 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:07.711838 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf36fd-9742-4474-97b9-cd5abb641d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:07.712129 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf36fd-9742-4474-97b9-cd5abb641d40 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:07.712778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf36fd-9742-4474-97b9-cd5abb641d40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:07.714735 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf36fd-9742-4474-97b9-cd5abb641d40 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:07.786778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 652/1308] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:07 2022] POST /identity/v3/users => generated 300 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:07.790300 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf36fd-9742-4474-97b9-cd5abb641d40 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:07.809001 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf36fd-9742-4474-97b9-cd5abb641d40 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:07.825831 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed57e0-1ed2-4c06-89e4-8ba76307ced4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:07.826686 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed57e0-1ed2-4c06-89e4-8ba76307ced4 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:07.827851 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed57e0-1ed2-4c06-89e4-8ba76307ced4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:07.828328 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed57e0-1ed2-4c06-89e4-8ba76307ced4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:07.829209 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed57e0-1ed2-4c06-89e4-8ba76307ced4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:07.831001 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed57e0-1ed2-4c06-89e4-8ba76307ced4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:07.863619 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 653/1309] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:48:07.867755 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed57e0-1ed2-4c06-89e4-8ba76307ced4 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:07.884738 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d04644-7cc4-460c-9b6a-61eea71ff187 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:07.890775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed57e0-1ed2-4c06-89e4-8ba76307ced4 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:07.903757 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6ddb8-6dce-493e-a8c7-d28367e642c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:07.904124 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badbc65-9ddd-4686-9a6a-098e06eb5057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:07.904418 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6ddb8-6dce-493e-a8c7-d28367e642c1 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:07.904703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6ddb8-6dce-493e-a8c7-d28367e642c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:07.905974 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badbc65-9ddd-4686-9a6a-098e06eb5057 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:07.906269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6ddb8-6dce-493e-a8c7-d28367e642c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:07.906919 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badbc65-9ddd-4686-9a6a-098e06eb5057 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:07.907414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badbc65-9ddd-4686-9a6a-098e06eb5057 admin admin] PATH_INFO: `/v3/projects/8ec9a49ab21f4c579ce47d5b3ebcd8c7/users/d62addca978547aba1b8e92f0c4ae698/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:07.931494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6ddb8-6dce-493e-a8c7-d28367e642c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:07.933058 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6ddb8-6dce-493e-a8c7-d28367e642c1 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:07.948300 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badbc65-9ddd-4686-9a6a-098e06eb5057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec9a49ab21f4c579ce47d5b3ebcd8c7, user_id=d62addca978547aba1b8e92f0c4ae698, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:07.963147 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badbc65-9ddd-4686-9a6a-098e06eb5057 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:07.970372 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 657/1310] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:07 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:48:08.019394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 654/1311] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:07 2022] PUT /identity/v3/projects/8ec9a49ab21f4c579ce47d5b3ebcd8c7/users/d62addca978547aba1b8e92f0c4ae698/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:08.025683 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6ddb8-6dce-493e-a8c7-d28367e642c1 service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:08.051183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6ddb8-6dce-493e-a8c7-d28367e642c1 service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:08.083655 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580cff9-d8ae-40b6-a314-ecc83ff680ef None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:08.083655 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580cff9-d8ae-40b6-a314-ecc83ff680ef None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:08.083655 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580cff9-d8ae-40b6-a314-ecc83ff680ef None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:08.093799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 655/1312] 10.222.0.36 () {54 vars in 961 bytes} [Sun Oct 30 00:48:08 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:08.131050 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d85c4-9280-4b0b-93fd-ed410d73c98d None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:08.131908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d85c4-9280-4b0b-93fd-ed410d73c98d None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:08.132709 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d85c4-9280-4b0b-93fd-ed410d73c98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:08.197041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-3d4d85c4-9280-4b0b-93fd-ed410d73c98d None None] MFA Rules not processed for user `adbba7be39a64230bab160ce77ce7586`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:08.198719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-3d4d85c4-9280-4b0b-93fd-ed410d73c98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:08.232044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00679d3f-8428-4fc7-879a-c4ee7bfefaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:08.232044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00679d3f-8428-4fc7-879a-c4ee7bfefaaf admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:08.232044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00679d3f-8428-4fc7-879a-c4ee7bfefaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:08.232388 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00679d3f-8428-4fc7-879a-c4ee7bfefaaf admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:08.232991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00679d3f-8428-4fc7-879a-c4ee7bfefaaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:08.236003 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00679d3f-8428-4fc7-879a-c4ee7bfefaaf admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:08.258814 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 658/1313] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:08.263898 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00679d3f-8428-4fc7-879a-c4ee7bfefaaf admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:08.298860 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00679d3f-8428-4fc7-879a-c4ee7bfefaaf admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:08.313762 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9560c-b20b-4105-a0bd-de1f60388a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:08.315287 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9560c-b20b-4105-a0bd-de1f60388a76 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:08.315615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9560c-b20b-4105-a0bd-de1f60388a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:08.316433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9560c-b20b-4105-a0bd-de1f60388a76 admin admin] PATH_INFO: `/v3/projects/8ec9a49ab21f4c579ce47d5b3ebcd8c7/users/d62addca978547aba1b8e92f0c4ae698/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:08.319159 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e9560c-b20b-4105-a0bd-de1f60388a76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec9a49ab21f4c579ce47d5b3ebcd8c7, user_id=d62addca978547aba1b8e92f0c4ae698, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:08.322895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e9560c-b20b-4105-a0bd-de1f60388a76 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:08.338539 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 659/1314] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:08 2022] PUT /identity/v3/projects/8ec9a49ab21f4c579ce47d5b3ebcd8c7/users/d62addca978547aba1b8e92f0c4ae698/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:08.353422 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14068119-2869-44ba-99ef-aa11bfd14119 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:08.354859 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14068119-2869-44ba-99ef-aa11bfd14119 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:08.354859 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14068119-2869-44ba-99ef-aa11bfd14119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:08.438576 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 656/1315] 10.222.0.36 () {58 vars in 1058 bytes} [Sun Oct 30 00:48:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:48:08.440641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d85c4-9280-4b0b-93fd-ed410d73c98d None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:08.461686 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d85c4-9280-4b0b-93fd-ed410d73c98d None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:08.465131 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-14068119-2869-44ba-99ef-aa11bfd14119 None None] MFA Rules not processed for user `d62addca978547aba1b8e92f0c4ae698`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:08.466449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-14068119-2869-44ba-99ef-aa11bfd14119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:08.568697 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:08.599660 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:08.610815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35e10b-9869-42d2-8c5c-7ed1725a9330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:08.612573 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35e10b-9869-42d2-8c5c-7ed1725a9330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:08.613127 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35e10b-9869-42d2-8c5c-7ed1725a9330 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:08.613508 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35e10b-9869-42d2-8c5c-7ed1725a9330 admin admin] PATH_INFO: `/v3/users/1557eb6485d345d0b7b54d4b32b1d77b` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:08.615453 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35e10b-9869-42d2-8c5c-7ed1725a9330 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1557eb6485d345d0b7b54d4b32b1d77b)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:08.621016 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: WARNING py.warnings [None req-fa35e10b-9869-42d2-8c5c-7ed1725a9330 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:48:08.621016 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: warnings.warn(msg) Oct 30 00:48:08.621016 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: Oct 30 00:48:08.623745 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35e10b-9869-42d2-8c5c-7ed1725a9330 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:08.646836 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 660/1316] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:08 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:08.652312 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a2cfd-3064-485c-818b-698d8d06971b None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:08.652817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a2cfd-3064-485c-818b-698d8d06971b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:08.653262 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a2cfd-3064-485c-818b-698d8d06971b None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:08.654754 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 661/1317] 10.222.0.36 () {54 vars in 912 bytes} [Sun Oct 30 00:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:08.656436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a2cfd-3064-485c-818b-698d8d06971b None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:08.724713 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-fa35e10b-9869-42d2-8c5c-7ed1725a9330 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1557eb6485d345d0b7b54d4b32b1d77b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:08.726587 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-fa35e10b-9869-42d2-8c5c-7ed1725a9330 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1557eb6485d345d0b7b54d4b32b1d77b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:08.769072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-fa35e10b-9869-42d2-8c5c-7ed1725a9330 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1557eb6485d345d0b7b54d4b32b1d77b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:08.783597 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-fa35e10b-9869-42d2-8c5c-7ed1725a9330 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1557eb6485d345d0b7b54d4b32b1d77b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:08.788239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a2cfd-3064-485c-818b-698d8d06971b None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:08.797952 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-fa35e10b-9869-42d2-8c5c-7ed1725a9330 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1557eb6485d345d0b7b54d4b32b1d77b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:08.801877 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca154c6d-f1d3-4f05-907b-94456a59e939 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:08.802542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca154c6d-f1d3-4f05-907b-94456a59e939 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:08.804981 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca154c6d-f1d3-4f05-907b-94456a59e939 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:08.805289 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca154c6d-f1d3-4f05-907b-94456a59e939 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:08.810591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-ca154c6d-f1d3-4f05-907b-94456a59e939 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:08.815084 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 657/1318] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:48:08 2022] DELETE /identity/v3/users/1557eb6485d345d0b7b54d4b32b1d77b => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:08.817075 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64416a1-7793-4b74-a6a1-72773a07cef2 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:08.818545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64416a1-7793-4b74-a6a1-72773a07cef2 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:08.818756 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64416a1-7793-4b74-a6a1-72773a07cef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:08.843715 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca154c6d-f1d3-4f05-907b-94456a59e939 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:08.847010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca154c6d-f1d3-4f05-907b-94456a59e939 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:08.895987 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-c64416a1-7793-4b74-a6a1-72773a07cef2 None None] MFA Rules not processed for user `adbba7be39a64230bab160ce77ce7586`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:08.896914 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c64416a1-7793-4b74-a6a1-72773a07cef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:08.910601 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 662/1319] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:08 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:08.916301 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa943ff-6522-4c65-b2ca-61d8cf5892a1 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:08.917166 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa943ff-6522-4c65-b2ca-61d8cf5892a1 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:08.919122 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa943ff-6522-4c65-b2ca-61d8cf5892a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:09.011596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-dfa943ff-6522-4c65-b2ca-61d8cf5892a1 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:09.013455 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-dfa943ff-6522-4c65-b2ca-61d8cf5892a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:09.084605 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 658/1320] 10.222.0.36 () {58 vars in 1009 bytes} [Sun Oct 30 00:48:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:09.092823 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64416a1-7793-4b74-a6a1-72773a07cef2 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:09.094645 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c64416a1-7793-4b74-a6a1-72773a07cef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:09.200165 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 663/1321] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:09.213501 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa943ff-6522-4c65-b2ca-61d8cf5892a1 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:09.219616 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-dfa943ff-6522-4c65-b2ca-61d8cf5892a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:09.242331 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64416a1-7793-4b74-a6a1-72773a07cef2 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:09.248067 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c675c38-0f33-411b-a8eb-4451f6014c7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:09.248675 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c675c38-0f33-411b-a8eb-4451f6014c7d service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:09.249559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c675c38-0f33-411b-a8eb-4451f6014c7d service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:09.249832 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c675c38-0f33-411b-a8eb-4451f6014c7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:09.274593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c675c38-0f33-411b-a8eb-4451f6014c7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:09.279337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c675c38-0f33-411b-a8eb-4451f6014c7d service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:09.313112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 659/1322] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:09.342687 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa943ff-6522-4c65-b2ca-61d8cf5892a1 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:09.348275 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ccf50-2a88-4e74-bc2a-bff32ae58ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:09.348967 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ccf50-2a88-4e74-bc2a-bff32ae58ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:09.350462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ccf50-2a88-4e74-bc2a-bff32ae58ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:09.350947 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ccf50-2a88-4e74-bc2a-bff32ae58ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:09.368324 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ccf50-2a88-4e74-bc2a-bff32ae58ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:09.370556 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ccf50-2a88-4e74-bc2a-bff32ae58ac7 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:09.392210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 664/1323] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:10.207059 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c675c38-0f33-411b-a8eb-4451f6014c7d service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:10.209307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4c675c38-0f33-411b-a8eb-4451f6014c7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:10.242264 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c675c38-0f33-411b-a8eb-4451f6014c7d service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:10.261689 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118c100-b63e-4aca-8995-a4198159ed93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:10.262586 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118c100-b63e-4aca-8995-a4198159ed93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:10.262747 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118c100-b63e-4aca-8995-a4198159ed93 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:10.262894 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118c100-b63e-4aca-8995-a4198159ed93 admin admin] PATH_INFO: `/v3/users/6290ef6f826d40c7a975be6eae0f1ec5` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:10.265812 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f118c100-b63e-4aca-8995-a4198159ed93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6290ef6f826d40c7a975be6eae0f1ec5)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:10.267801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f118c100-b63e-4aca-8995-a4198159ed93 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:10.272634 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:10.367774 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f118c100-b63e-4aca-8995-a4198159ed93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6290ef6f826d40c7a975be6eae0f1ec5'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:10.368740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f118c100-b63e-4aca-8995-a4198159ed93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6290ef6f826d40c7a975be6eae0f1ec5'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:10.385910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f118c100-b63e-4aca-8995-a4198159ed93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6290ef6f826d40c7a975be6eae0f1ec5'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:10.401018 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f118c100-b63e-4aca-8995-a4198159ed93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6290ef6f826d40c7a975be6eae0f1ec5'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:10.413927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f118c100-b63e-4aca-8995-a4198159ed93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6290ef6f826d40c7a975be6eae0f1ec5'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:10.428165 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 660/1324] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:48:10 2022] DELETE /identity/v3/users/6290ef6f826d40c7a975be6eae0f1ec5 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:10.433767 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff166d79-ba28-42eb-a066-1e109434d032 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:10.434850 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff166d79-ba28-42eb-a066-1e109434d032 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:10.435442 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff166d79-ba28-42eb-a066-1e109434d032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:10.486270 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-ff166d79-ba28-42eb-a066-1e109434d032 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:10.487523 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-ff166d79-ba28-42eb-a066-1e109434d032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:10.523982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:10.634553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 665/1325] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:48:10.782367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118c100-b63e-4aca-8995-a4198159ed93 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:10.784995 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f118c100-b63e-4aca-8995-a4198159ed93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:10.968678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff166d79-ba28-42eb-a066-1e109434d032 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:10.970792 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-ff166d79-ba28-42eb-a066-1e109434d032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:10.995931 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff166d79-ba28-42eb-a066-1e109434d032 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:11.001784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae996206-90a2-465d-a81d-0867cc8bbb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:11.002441 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae996206-90a2-465d-a81d-0867cc8bbb34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:11.002723 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae996206-90a2-465d-a81d-0867cc8bbb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:11.003001 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae996206-90a2-465d-a81d-0867cc8bbb34 admin admin] PATH_INFO: `/v3/users/220cbd96393049598539054b1d2a180e` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:11.012893 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae996206-90a2-465d-a81d-0867cc8bbb34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=220cbd96393049598539054b1d2a180e)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:11.015374 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae996206-90a2-465d-a81d-0867cc8bbb34 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:11.054692 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118c100-b63e-4aca-8995-a4198159ed93 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:11.063259 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f8af5-89fb-4bc2-9ae1-f1091ed99685 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:11.064166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f8af5-89fb-4bc2-9ae1-f1091ed99685 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:11.064883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f8af5-89fb-4bc2-9ae1-f1091ed99685 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:11.065405 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f8af5-89fb-4bc2-9ae1-f1091ed99685 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:11.087601 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f8af5-89fb-4bc2-9ae1-f1091ed99685 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:11.090925 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f8af5-89fb-4bc2-9ae1-f1091ed99685 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:11.119094 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 661/1326] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:11.121862 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-ae996206-90a2-465d-a81d-0867cc8bbb34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '220cbd96393049598539054b1d2a180e'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:11.122730 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-ae996206-90a2-465d-a81d-0867cc8bbb34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '220cbd96393049598539054b1d2a180e'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:11.153068 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-ae996206-90a2-465d-a81d-0867cc8bbb34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '220cbd96393049598539054b1d2a180e'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:11.171446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-ae996206-90a2-465d-a81d-0867cc8bbb34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '220cbd96393049598539054b1d2a180e'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:11.173582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-ae996206-90a2-465d-a81d-0867cc8bbb34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '220cbd96393049598539054b1d2a180e'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:11.199598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 666/1327] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:48:10 2022] DELETE /identity/v3/users/220cbd96393049598539054b1d2a180e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:11.206868 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff59e13-6361-4307-bb27-179fe546828b None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:11.209967 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff59e13-6361-4307-bb27-179fe546828b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:11.211885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff59e13-6361-4307-bb27-179fe546828b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:11.246970 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-5ff59e13-6361-4307-bb27-179fe546828b None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:11.250868 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-5ff59e13-6361-4307-bb27-179fe546828b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:11.404131 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 662/1328] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:11.415976 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae996206-90a2-465d-a81d-0867cc8bbb34 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:11.418168 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-ae996206-90a2-465d-a81d-0867cc8bbb34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:11.649107 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae996206-90a2-465d-a81d-0867cc8bbb34 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:11.663989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7592ec-4830-4539-b51f-b22208caa939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:11.664661 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7592ec-4830-4539-b51f-b22208caa939 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:11.664978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7592ec-4830-4539-b51f-b22208caa939 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:11.665260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7592ec-4830-4539-b51f-b22208caa939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:11.687590 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7592ec-4830-4539-b51f-b22208caa939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:11.688696 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7592ec-4830-4539-b51f-b22208caa939 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:11.716409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 667/1329] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:12.927341 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff59e13-6361-4307-bb27-179fe546828b None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:12.929575 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-5ff59e13-6361-4307-bb27-179fe546828b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:12.959879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff59e13-6361-4307-bb27-179fe546828b None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:12.970555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bedd19-e746-4956-a8bb-6d0bca3d4918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:12.971674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bedd19-e746-4956-a8bb-6d0bca3d4918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:12.972246 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bedd19-e746-4956-a8bb-6d0bca3d4918 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:12.972787 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bedd19-e746-4956-a8bb-6d0bca3d4918 admin admin] PATH_INFO: `/v3/projects/7d10f71e30d04db39566e439f4fba6a1` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:12.974960 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bedd19-e746-4956-a8bb-6d0bca3d4918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d10f71e30d04db39566e439f4fba6a1)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:12.977259 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bedd19-e746-4956-a8bb-6d0bca3d4918 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:12.995127 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7592ec-4830-4539-b51f-b22208caa939 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:12.999758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-dd7592ec-4830-4539-b51f-b22208caa939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:13.027083 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7592ec-4830-4539-b51f-b22208caa939 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:13.035731 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f19502-eba0-4498-acf0-8d1111459f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:13.036706 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f19502-eba0-4498-acf0-8d1111459f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:13.037140 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f19502-eba0-4498-acf0-8d1111459f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:13.037481 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f19502-eba0-4498-acf0-8d1111459f90 admin admin] PATH_INFO: `/v3/projects/39791c410e0e495e94e692fda6bd5e23` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:13.038746 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f19502-eba0-4498-acf0-8d1111459f90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39791c410e0e495e94e692fda6bd5e23)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:13.040656 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: WARNING py.warnings [None req-f2f19502-eba0-4498-acf0-8d1111459f90 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 30 00:48:13.040656 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: warnings.warn(msg) Oct 30 00:48:13.040656 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: Oct 30 00:48:13.043513 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f19502-eba0-4498-acf0-8d1111459f90 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:13.049057 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:13.126512 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-60bedd19-e746-4956-a8bb-6d0bca3d4918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d10f71e30d04db39566e439f4fba6a1', 'request_id': 'req-60bedd19-e746-4956-a8bb-6d0bca3d4918'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:13.150948 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-60bedd19-e746-4956-a8bb-6d0bca3d4918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d10f71e30d04db39566e439f4fba6a1', 'request_id': 'req-60bedd19-e746-4956-a8bb-6d0bca3d4918'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:13.155827 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:13.163297 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-60bedd19-e746-4956-a8bb-6d0bca3d4918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d10f71e30d04db39566e439f4fba6a1', 'request_id': 'req-60bedd19-e746-4956-a8bb-6d0bca3d4918'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:13.190619 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f2f19502-eba0-4498-acf0-8d1111459f90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39791c410e0e495e94e692fda6bd5e23', 'request_id': 'req-f2f19502-eba0-4498-acf0-8d1111459f90'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:13.198839 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-60bedd19-e746-4956-a8bb-6d0bca3d4918 admin admin] The token cache is being invalidate because project 7d10f71e30d04db39566e439f4fba6a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:48:13.198839 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-60bedd19-e746-4956-a8bb-6d0bca3d4918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:13.199955 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-60bedd19-e746-4956-a8bb-6d0bca3d4918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:13.206591 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 663/1330] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:48:12 2022] DELETE /identity/v3/projects/7d10f71e30d04db39566e439f4fba6a1 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:13.209764 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3408d9-ed5a-4a9f-8960-20e412f91ea5 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:13.210217 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3408d9-ed5a-4a9f-8960-20e412f91ea5 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:13.210669 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3408d9-ed5a-4a9f-8960-20e412f91ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:13.215164 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f2f19502-eba0-4498-acf0-8d1111459f90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39791c410e0e495e94e692fda6bd5e23', 'request_id': 'req-f2f19502-eba0-4498-acf0-8d1111459f90'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:13.242177 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-de3408d9-ed5a-4a9f-8960-20e412f91ea5 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:13.243677 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f2f19502-eba0-4498-acf0-8d1111459f90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39791c410e0e495e94e692fda6bd5e23', 'request_id': 'req-f2f19502-eba0-4498-acf0-8d1111459f90'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:13.243820 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-de3408d9-ed5a-4a9f-8960-20e412f91ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:13.278528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f2f19502-eba0-4498-acf0-8d1111459f90 admin admin] The token cache is being invalidate because project 39791c410e0e495e94e692fda6bd5e23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:48:13.278813 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f2f19502-eba0-4498-acf0-8d1111459f90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:13.279483 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f2f19502-eba0-4498-acf0-8d1111459f90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:13.281611 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 668/1331] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:48:12 2022] DELETE /identity/v3/projects/39791c410e0e495e94e692fda6bd5e23 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:13.285360 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f19502-eba0-4498-acf0-8d1111459f90 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:13.286810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-f2f19502-eba0-4498-acf0-8d1111459f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:13.495654 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 664/1332] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:13.499513 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f19502-eba0-4498-acf0-8d1111459f90 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:13.503815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8482c-976a-4d4e-a7da-dc69480c1ef8 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:13.504136 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8482c-976a-4d4e-a7da-dc69480c1ef8 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:13.504425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8482c-976a-4d4e-a7da-dc69480c1ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:13.523916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb1457-ebb9-4f32-b2ba-05e1eb2c814a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:13.524665 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb1457-ebb9-4f32-b2ba-05e1eb2c814a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:13.524955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb1457-ebb9-4f32-b2ba-05e1eb2c814a admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:13.525546 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb1457-ebb9-4f32-b2ba-05e1eb2c814a admin admin] PATH_INFO: `/v3/users/f03c979471d64e1d9638a9c4eef1f8e1` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:13.529316 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb1457-ebb9-4f32-b2ba-05e1eb2c814a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f03c979471d64e1d9638a9c4eef1f8e1)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:13.533601 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb1457-ebb9-4f32-b2ba-05e1eb2c814a admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:13.573222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-38e8482c-976a-4d4e-a7da-dc69480c1ef8 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:13.577833 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-38e8482c-976a-4d4e-a7da-dc69480c1ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:13.606851 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 665/1333] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:13.610695 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8482c-976a-4d4e-a7da-dc69480c1ef8 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:13.649236 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8482c-976a-4d4e-a7da-dc69480c1ef8 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:13.665873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462f178-dbec-4515-b4a8-c087e3c0a7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:13.666059 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f7eb1457-ebb9-4f32-b2ba-05e1eb2c814a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f03c979471d64e1d9638a9c4eef1f8e1'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:13.666529 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462f178-dbec-4515-b4a8-c087e3c0a7f2 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:13.666817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462f178-dbec-4515-b4a8-c087e3c0a7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:13.667096 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462f178-dbec-4515-b4a8-c087e3c0a7f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:13.667767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462f178-dbec-4515-b4a8-c087e3c0a7f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:13.667879 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f7eb1457-ebb9-4f32-b2ba-05e1eb2c814a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f03c979471d64e1d9638a9c4eef1f8e1'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:13.673278 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462f178-dbec-4515-b4a8-c087e3c0a7f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:13.675116 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462f178-dbec-4515-b4a8-c087e3c0a7f2 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:13.702225 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f7eb1457-ebb9-4f32-b2ba-05e1eb2c814a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f03c979471d64e1d9638a9c4eef1f8e1'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:13.713141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f7eb1457-ebb9-4f32-b2ba-05e1eb2c814a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f03c979471d64e1d9638a9c4eef1f8e1'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:13.716825 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f7eb1457-ebb9-4f32-b2ba-05e1eb2c814a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f03c979471d64e1d9638a9c4eef1f8e1'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:13.728986 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 666/1334] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:13.731248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462f178-dbec-4515-b4a8-c087e3c0a7f2 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:13.736365 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-2462f178-dbec-4515-b4a8-c087e3c0a7f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:13.765642 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 669/1335] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:48:13 2022] DELETE /identity/v3/users/f03c979471d64e1d9638a9c4eef1f8e1 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:48:13.770771 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462f178-dbec-4515-b4a8-c087e3c0a7f2 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:13.908607 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b68d4-9ac9-43a6-99dd-d50719d440ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:13.909250 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b68d4-9ac9-43a6-99dd-d50719d440ce admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:13.909537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b68d4-9ac9-43a6-99dd-d50719d440ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:13.909818 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b68d4-9ac9-43a6-99dd-d50719d440ce admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:13.910469 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b68d4-9ac9-43a6-99dd-d50719d440ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:13.912555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b68d4-9ac9-43a6-99dd-d50719d440ce admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:13.978874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 667/1336] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:13 2022] POST /identity/v3/projects => generated 373 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:13.982760 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb1457-ebb9-4f32-b2ba-05e1eb2c814a admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:14.012812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb1457-ebb9-4f32-b2ba-05e1eb2c814a admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:14.113831 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:14.141056 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488856c6-e877-4d03-a597-120e45bd2ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:14.142215 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488856c6-e877-4d03-a597-120e45bd2ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:14.143965 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488856c6-e877-4d03-a597-120e45bd2ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:14.144416 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488856c6-e877-4d03-a597-120e45bd2ff5 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:14.145238 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488856c6-e877-4d03-a597-120e45bd2ff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:14.147562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488856c6-e877-4d03-a597-120e45bd2ff5 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:14.185814 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 670/1337] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:13 2022] POST /identity/v3/users => generated 287 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:14.191064 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b68d4-9ac9-43a6-99dd-d50719d440ce admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:14.206180 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b68d4-9ac9-43a6-99dd-d50719d440ce admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:14.217402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6e693-77f3-4654-8780-a11211d46512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:14.218610 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6e693-77f3-4654-8780-a11211d46512 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:14.219003 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6e693-77f3-4654-8780-a11211d46512 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:14.219278 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6e693-77f3-4654-8780-a11211d46512 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:14.221422 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad6e693-77f3-4654-8780-a11211d46512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:14.226413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad6e693-77f3-4654-8780-a11211d46512 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:14.262561 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 668/1338] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:14.266914 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488856c6-e877-4d03-a597-120e45bd2ff5 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:14.268932 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-488856c6-e877-4d03-a597-120e45bd2ff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:14.277602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6e693-77f3-4654-8780-a11211d46512 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:14.310590 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:14.316827 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488856c6-e877-4d03-a597-120e45bd2ff5 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:14.330119 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6e693-77f3-4654-8780-a11211d46512 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:14.335420 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f10c7-e092-4ff3-8080-fc78b92200d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:14.336131 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f10c7-e092-4ff3-8080-fc78b92200d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:14.336422 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f10c7-e092-4ff3-8080-fc78b92200d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:14.339629 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f10c7-e092-4ff3-8080-fc78b92200d2 admin admin] PATH_INFO: `/v3/projects/3a9494126a7b463c801ff049f0241b09` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:14.346101 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f10c7-e092-4ff3-8080-fc78b92200d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a9494126a7b463c801ff049f0241b09)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:14.348687 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f10c7-e092-4ff3-8080-fc78b92200d2 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:14.352892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550b7e8-7f96-4269-a6fa-41e218f98f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:14.353531 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550b7e8-7f96-4269-a6fa-41e218f98f25 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:14.353814 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550b7e8-7f96-4269-a6fa-41e218f98f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:14.354088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550b7e8-7f96-4269-a6fa-41e218f98f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:14.357203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550b7e8-7f96-4269-a6fa-41e218f98f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:14.359549 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550b7e8-7f96-4269-a6fa-41e218f98f25 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:14.399576 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 669/1339] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:14.407906 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550b7e8-7f96-4269-a6fa-41e218f98f25 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:14.455225 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550b7e8-7f96-4269-a6fa-41e218f98f25 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:14.478892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de7c48-06a3-4b97-8168-cbd3c5e37c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:14.478892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de7c48-06a3-4b97-8168-cbd3c5e37c95 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:14.478892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de7c48-06a3-4b97-8168-cbd3c5e37c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:14.478892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de7c48-06a3-4b97-8168-cbd3c5e37c95 admin admin] PATH_INFO: `/v3/projects/10ae58bd73364dc0965b6ab60119ef53/users/41613d6504334fd6aca800fe9ee6526b/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:14.526350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-087f10c7-e092-4ff3-8080-fc78b92200d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a9494126a7b463c801ff049f0241b09', 'request_id': 'req-087f10c7-e092-4ff3-8080-fc78b92200d2'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:14.532394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de7c48-06a3-4b97-8168-cbd3c5e37c95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10ae58bd73364dc0965b6ab60119ef53, user_id=41613d6504334fd6aca800fe9ee6526b, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:14.537342 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de7c48-06a3-4b97-8168-cbd3c5e37c95 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:14.551953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-087f10c7-e092-4ff3-8080-fc78b92200d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a9494126a7b463c801ff049f0241b09', 'request_id': 'req-087f10c7-e092-4ff3-8080-fc78b92200d2'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:14.579827 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-087f10c7-e092-4ff3-8080-fc78b92200d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a9494126a7b463c801ff049f0241b09', 'request_id': 'req-087f10c7-e092-4ff3-8080-fc78b92200d2'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:14.606987 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 670/1340] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:14 2022] PUT /identity/v3/projects/10ae58bd73364dc0965b6ab60119ef53/users/41613d6504334fd6aca800fe9ee6526b/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:14.609872 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000594a3-2c32-46f7-a454-766a25627858 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:14.610161 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000594a3-2c32-46f7-a454-766a25627858 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:14.610437 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000594a3-2c32-46f7-a454-766a25627858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:14.641620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-087f10c7-e092-4ff3-8080-fc78b92200d2 admin admin] The token cache is being invalidate because project 3a9494126a7b463c801ff049f0241b09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:48:14.641948 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-087f10c7-e092-4ff3-8080-fc78b92200d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:14.644245 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-087f10c7-e092-4ff3-8080-fc78b92200d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:14.654105 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 671/1341] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:48:14 2022] DELETE /identity/v3/projects/3a9494126a7b463c801ff049f0241b09 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:14.655636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f10c7-e092-4ff3-8080-fc78b92200d2 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:14.667693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-087f10c7-e092-4ff3-8080-fc78b92200d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:14.694943 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-000594a3-2c32-46f7-a454-766a25627858 None None] MFA Rules not processed for user `41613d6504334fd6aca800fe9ee6526b`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:14.697754 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-000594a3-2c32-46f7-a454-766a25627858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:14.916863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f10c7-e092-4ff3-8080-fc78b92200d2 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:15.011078 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 671/1342] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:15.016444 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000594a3-2c32-46f7-a454-766a25627858 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:15.017889 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-000594a3-2c32-46f7-a454-766a25627858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:15.066724 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd181297-82f6-4a99-83ad-1edc593d263d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:15.067872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd181297-82f6-4a99-83ad-1edc593d263d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:15.068534 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd181297-82f6-4a99-83ad-1edc593d263d admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:15.068953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd181297-82f6-4a99-83ad-1edc593d263d admin admin] PATH_INFO: `/v3/users/d62addca978547aba1b8e92f0c4ae698` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:15.071102 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd181297-82f6-4a99-83ad-1edc593d263d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d62addca978547aba1b8e92f0c4ae698)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:15.073448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd181297-82f6-4a99-83ad-1edc593d263d admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:15.077877 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:15.173079 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-dd181297-82f6-4a99-83ad-1edc593d263d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd62addca978547aba1b8e92f0c4ae698'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:15.173843 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-dd181297-82f6-4a99-83ad-1edc593d263d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd62addca978547aba1b8e92f0c4ae698'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:15.198727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-dd181297-82f6-4a99-83ad-1edc593d263d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd62addca978547aba1b8e92f0c4ae698'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:15.206441 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000594a3-2c32-46f7-a454-766a25627858 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:15.213146 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-dd181297-82f6-4a99-83ad-1edc593d263d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd62addca978547aba1b8e92f0c4ae698'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:15.214513 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-dd181297-82f6-4a99-83ad-1edc593d263d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd62addca978547aba1b8e92f0c4ae698'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:15.216475 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2786308-b593-4d6f-a15b-04ed4a56cdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:15.217115 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2786308-b593-4d6f-a15b-04ed4a56cdfa admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:15.217947 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2786308-b593-4d6f-a15b-04ed4a56cdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:15.218481 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2786308-b593-4d6f-a15b-04ed4a56cdfa admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:15.219540 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2786308-b593-4d6f-a15b-04ed4a56cdfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:15.223447 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2786308-b593-4d6f-a15b-04ed4a56cdfa admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:15.235593 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 672/1343] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:48:14 2022] DELETE /identity/v3/users/d62addca978547aba1b8e92f0c4ae698 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:15.239741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab5872-c581-4229-ac80-83419b34553c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:15.240156 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab5872-c581-4229-ac80-83419b34553c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:15.240533 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab5872-c581-4229-ac80-83419b34553c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:15.319000 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 672/1344] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:15 2022] POST /identity/v3/projects => generated 371 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:15.322785 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2786308-b593-4d6f-a15b-04ed4a56cdfa admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:15.322979 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-a6ab5872-c581-4229-ac80-83419b34553c None None] MFA Rules not processed for user `df1ee15eb3bc468188a9d8c1af46529c`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:15.324255 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a6ab5872-c581-4229-ac80-83419b34553c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:15.325534 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e2786308-b593-4d6f-a15b-04ed4a56cdfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:15.441226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:15.476822 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 673/1345] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:15.485417 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2786308-b593-4d6f-a15b-04ed4a56cdfa admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:15.492156 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fcac15-3dec-4855-b2a5-4bc5c457708e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:15.493505 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fcac15-3dec-4855-b2a5-4bc5c457708e admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:15.493941 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fcac15-3dec-4855-b2a5-4bc5c457708e admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:15.494225 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fcac15-3dec-4855-b2a5-4bc5c457708e admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:15.497108 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fcac15-3dec-4855-b2a5-4bc5c457708e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:15.499731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fcac15-3dec-4855-b2a5-4bc5c457708e admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:15.506143 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab5872-c581-4229-ac80-83419b34553c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:15.507692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a6ab5872-c581-4229-ac80-83419b34553c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:15.541438 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 673/1346] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:15 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:48:15.542430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fcac15-3dec-4855-b2a5-4bc5c457708e admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:15.567914 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fcac15-3dec-4855-b2a5-4bc5c457708e admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:15.574938 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045feec0-8058-4c7d-a508-a160e9818022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:15.575871 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045feec0-8058-4c7d-a508-a160e9818022 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:15.579809 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045feec0-8058-4c7d-a508-a160e9818022 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:15.581141 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045feec0-8058-4c7d-a508-a160e9818022 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:15.581501 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045feec0-8058-4c7d-a508-a160e9818022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:15.583221 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045feec0-8058-4c7d-a508-a160e9818022 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:15.616918 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 674/1347] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:15.618453 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045feec0-8058-4c7d-a508-a160e9818022 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:15.642458 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045feec0-8058-4c7d-a508-a160e9818022 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:15.652843 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f409ca1-385c-4e80-9364-8382c87883d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:15.653701 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f409ca1-385c-4e80-9364-8382c87883d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:15.653985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f409ca1-385c-4e80-9364-8382c87883d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:15.654257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f409ca1-385c-4e80-9364-8382c87883d7 admin admin] PATH_INFO: `/v3/projects/82297cb747944853a03350f2aed55bb1/users/ad494471d3304ff393efe693b0b52524/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:15.703922 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f409ca1-385c-4e80-9364-8382c87883d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82297cb747944853a03350f2aed55bb1, user_id=ad494471d3304ff393efe693b0b52524, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:15.708224 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f409ca1-385c-4e80-9364-8382c87883d7 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:15.741993 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 675/1348] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:15 2022] PUT /identity/v3/projects/82297cb747944853a03350f2aed55bb1/users/ad494471d3304ff393efe693b0b52524/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:15.746231 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f409ca1-385c-4e80-9364-8382c87883d7 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:15.752996 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab5872-c581-4229-ac80-83419b34553c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:15.770954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:16.011266 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0f806-f435-4110-85c6-d44c75b3382a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:16.012267 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0f806-f435-4110-85c6-d44c75b3382a service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:16.012745 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0f806-f435-4110-85c6-d44c75b3382a service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:16.013175 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0f806-f435-4110-85c6-d44c75b3382a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:16.056679 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f409ca1-385c-4e80-9364-8382c87883d7 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:16.095569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677a062-20d4-46eb-b9f3-8100a4233204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:16.096262 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677a062-20d4-46eb-b9f3-8100a4233204 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:16.096548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677a062-20d4-46eb-b9f3-8100a4233204 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:16.096823 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677a062-20d4-46eb-b9f3-8100a4233204 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:16.097490 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677a062-20d4-46eb-b9f3-8100a4233204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:16.099582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677a062-20d4-46eb-b9f3-8100a4233204 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:16.121099 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 676/1349] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:16.127383 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677a062-20d4-46eb-b9f3-8100a4233204 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:16.130043 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c677a062-20d4-46eb-b9f3-8100a4233204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:16.145620 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677a062-20d4-46eb-b9f3-8100a4233204 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:16.151290 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b45b5e-dbc9-47ae-af2c-fd6bc4931379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:16.152964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b45b5e-dbc9-47ae-af2c-fd6bc4931379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:16.153934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b45b5e-dbc9-47ae-af2c-fd6bc4931379 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:16.157331 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b45b5e-dbc9-47ae-af2c-fd6bc4931379 admin admin] PATH_INFO: `/v3/projects/8ec9a49ab21f4c579ce47d5b3ebcd8c7` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:16.159193 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b45b5e-dbc9-47ae-af2c-fd6bc4931379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ec9a49ab21f4c579ce47d5b3ebcd8c7)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:16.161418 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b45b5e-dbc9-47ae-af2c-fd6bc4931379 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:16.184560 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0f806-f435-4110-85c6-d44c75b3382a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:16.191767 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0f806-f435-4110-85c6-d44c75b3382a service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:16.219326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 674/1350] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 715 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:48:16.221895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0f806-f435-4110-85c6-d44c75b3382a service nova] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:16.250514 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0f806-f435-4110-85c6-d44c75b3382a service nova] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:16.312479 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-28b45b5e-dbc9-47ae-af2c-fd6bc4931379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ec9a49ab21f4c579ce47d5b3ebcd8c7', 'request_id': 'req-28b45b5e-dbc9-47ae-af2c-fd6bc4931379'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:16.335283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-28b45b5e-dbc9-47ae-af2c-fd6bc4931379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ec9a49ab21f4c579ce47d5b3ebcd8c7', 'request_id': 'req-28b45b5e-dbc9-47ae-af2c-fd6bc4931379'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:16.349945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-28b45b5e-dbc9-47ae-af2c-fd6bc4931379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ec9a49ab21f4c579ce47d5b3ebcd8c7', 'request_id': 'req-28b45b5e-dbc9-47ae-af2c-fd6bc4931379'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:16.369092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-28b45b5e-dbc9-47ae-af2c-fd6bc4931379 admin admin] The token cache is being invalidate because project 8ec9a49ab21f4c579ce47d5b3ebcd8c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:48:16.373410 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-28b45b5e-dbc9-47ae-af2c-fd6bc4931379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:16.375401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-28b45b5e-dbc9-47ae-af2c-fd6bc4931379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:16.377511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 677/1351] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:48:16 2022] DELETE /identity/v3/projects/8ec9a49ab21f4c579ce47d5b3ebcd8c7 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:16.382667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a50e4-0626-4b27-9f60-8e8a0928fa14 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:16.383066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a50e4-0626-4b27-9f60-8e8a0928fa14 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:16.383566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a50e4-0626-4b27-9f60-8e8a0928fa14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:16.402377 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07ae38-f4d7-4e87-ba68-8aeaa98d2477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:16.403466 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07ae38-f4d7-4e87-ba68-8aeaa98d2477 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:16.403994 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07ae38-f4d7-4e87-ba68-8aeaa98d2477 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:16.404510 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07ae38-f4d7-4e87-ba68-8aeaa98d2477 admin admin] PATH_INFO: `/v3/projects/82297cb747944853a03350f2aed55bb1/users/ad494471d3304ff393efe693b0b52524/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:16.412109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c07ae38-f4d7-4e87-ba68-8aeaa98d2477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82297cb747944853a03350f2aed55bb1, user_id=ad494471d3304ff393efe693b0b52524, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:16.417194 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c07ae38-f4d7-4e87-ba68-8aeaa98d2477 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:16.431689 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-3f9a50e4-0626-4b27-9f60-8e8a0928fa14 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:16.434557 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-3f9a50e4-0626-4b27-9f60-8e8a0928fa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:16.445966 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 675/1352] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:16 2022] PUT /identity/v3/projects/82297cb747944853a03350f2aed55bb1/users/ad494471d3304ff393efe693b0b52524/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:16.456966 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fabb6a-36c3-4ae3-b31f-cce6701736ae None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:16.457420 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fabb6a-36c3-4ae3-b31f-cce6701736ae None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:16.457923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fabb6a-36c3-4ae3-b31f-cce6701736ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:16.459118 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:16.553342 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-c2fabb6a-36c3-4ae3-b31f-cce6701736ae None None] MFA Rules not processed for user `ad494471d3304ff393efe693b0b52524`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:16.555368 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c2fabb6a-36c3-4ae3-b31f-cce6701736ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:16.573701 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 678/1353] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:16.576409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b015e-2a70-4df2-a6f6-f8e2be057480 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:16.577596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b015e-2a70-4df2-a6f6-f8e2be057480 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:16.577997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b015e-2a70-4df2-a6f6-f8e2be057480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:16.628341 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-3e2b015e-2a70-4df2-a6f6-f8e2be057480 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:16.633114 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-3e2b015e-2a70-4df2-a6f6-f8e2be057480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:16.643892 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:16.682165 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 679/1354] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:16.683722 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b015e-2a70-4df2-a6f6-f8e2be057480 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:16.705479 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b015e-2a70-4df2-a6f6-f8e2be057480 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:16.710967 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2d8c6-7a52-492c-a129-f8f542c23c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:16.711604 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2d8c6-7a52-492c-a129-f8f542c23c9e admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:16.711889 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2d8c6-7a52-492c-a129-f8f542c23c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:16.712199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2d8c6-7a52-492c-a129-f8f542c23c9e admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:16.713089 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2d8c6-7a52-492c-a129-f8f542c23c9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:16.713401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2d8c6-7a52-492c-a129-f8f542c23c9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:16.715401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2d8c6-7a52-492c-a129-f8f542c23c9e admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:16.742066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 680/1355] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:16.744552 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2d8c6-7a52-492c-a129-f8f542c23c9e admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:16.755252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 676/1356] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:48:16.772898 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fabb6a-36c3-4ae3-b31f-cce6701736ae None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:16.774692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c2fabb6a-36c3-4ae3-b31f-cce6701736ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:16.775929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2d8c6-7a52-492c-a129-f8f542c23c9e admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:16.788134 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d202fe-ef93-498e-807d-9200d771b8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:16.788821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d202fe-ef93-498e-807d-9200d771b8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:16.789104 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d202fe-ef93-498e-807d-9200d771b8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:16.789381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d202fe-ef93-498e-807d-9200d771b8c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:16.791716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d202fe-ef93-498e-807d-9200d771b8c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:16.800207 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d202fe-ef93-498e-807d-9200d771b8c3 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:16.860248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 681/1357] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:16 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:16.861679 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d202fe-ef93-498e-807d-9200d771b8c3 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:16.898222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fabb6a-36c3-4ae3-b31f-cce6701736ae None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:17.003459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d202fe-ef93-498e-807d-9200d771b8c3 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:17.003846 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e77e72-d64e-4b37-982d-4eed6eed3157 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:17.004530 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e77e72-d64e-4b37-982d-4eed6eed3157 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:17.004832 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e77e72-d64e-4b37-982d-4eed6eed3157 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:17.005080 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e77e72-d64e-4b37-982d-4eed6eed3157 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:17.012054 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2f44d-6bff-4857-b27b-834b7af30257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:17.012991 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2f44d-6bff-4857-b27b-834b7af30257 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:17.013472 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2f44d-6bff-4857-b27b-834b7af30257 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:17.013850 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2f44d-6bff-4857-b27b-834b7af30257 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:17.015174 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2f44d-6bff-4857-b27b-834b7af30257 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:17.017659 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2f44d-6bff-4857-b27b-834b7af30257 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:17.050073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 682/1358] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:16 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:48:17.053898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2f44d-6bff-4857-b27b-834b7af30257 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:17.069721 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2f44d-6bff-4857-b27b-834b7af30257 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:17.078186 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb5313-0fa8-428e-84a2-13916b8edae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:17.079032 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb5313-0fa8-428e-84a2-13916b8edae4 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:17.079660 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb5313-0fa8-428e-84a2-13916b8edae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:17.080560 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb5313-0fa8-428e-84a2-13916b8edae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:17.088705 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb5313-0fa8-428e-84a2-13916b8edae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:17.090921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb5313-0fa8-428e-84a2-13916b8edae4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:17.111846 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e77e72-d64e-4b37-982d-4eed6eed3157 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:17.113475 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e77e72-d64e-4b37-982d-4eed6eed3157 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:17.116998 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 683/1359] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:48:17.120863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb5313-0fa8-428e-84a2-13916b8edae4 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:17.137771 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 677/1360] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:17.151969 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb5313-0fa8-428e-84a2-13916b8edae4 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:17.159153 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16daf45-f516-4a54-985c-89e97087a0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:17.160810 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16daf45-f516-4a54-985c-89e97087a0ba admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:17.161489 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16daf45-f516-4a54-985c-89e97087a0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:17.162020 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16daf45-f516-4a54-985c-89e97087a0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:17.162930 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16daf45-f516-4a54-985c-89e97087a0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:17.167678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16daf45-f516-4a54-985c-89e97087a0ba admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:17.192047 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 684/1361] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:48:17.196146 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e77e72-d64e-4b37-982d-4eed6eed3157 service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:17.219830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e77e72-d64e-4b37-982d-4eed6eed3157 service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:17.225924 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba5032-d3f2-4679-832d-1f8136b906af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:17.228124 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16daf45-f516-4a54-985c-89e97087a0ba admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:17.230279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e16daf45-f516-4a54-985c-89e97087a0ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:17.235521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba5032-d3f2-4679-832d-1f8136b906af admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:17.238930 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba5032-d3f2-4679-832d-1f8136b906af admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:17.240409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba5032-d3f2-4679-832d-1f8136b906af admin admin] PATH_INFO: `/v3/projects/1e439515385741ca90d0429882c1f84b/users/b882814eba3f4ebcb5e735ea7bf429fb/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:17.256438 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16daf45-f516-4a54-985c-89e97087a0ba admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:17.269962 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a26b8-04ef-4bf7-b759-6f6c15c64acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:17.273185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a26b8-04ef-4bf7-b759-6f6c15c64acc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:17.273185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a26b8-04ef-4bf7-b759-6f6c15c64acc admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:17.274742 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a26b8-04ef-4bf7-b759-6f6c15c64acc admin admin] PATH_INFO: `/v3/projects/86f31b7a33f04f9d8b20794bfca91f10` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:17.277759 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a26b8-04ef-4bf7-b759-6f6c15c64acc admin admin] RBAC: Authorizing `identity:delete_project(project_id=86f31b7a33f04f9d8b20794bfca91f10)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:17.278778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba5032-d3f2-4679-832d-1f8136b906af admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e439515385741ca90d0429882c1f84b, user_id=b882814eba3f4ebcb5e735ea7bf429fb, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:17.284061 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba5032-d3f2-4679-832d-1f8136b906af admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:17.284828 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a26b8-04ef-4bf7-b759-6f6c15c64acc admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:17.316295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 678/1362] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:17 2022] PUT /identity/v3/projects/1e439515385741ca90d0429882c1f84b/users/b882814eba3f4ebcb5e735ea7bf429fb/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:17.328332 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba5032-d3f2-4679-832d-1f8136b906af admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:17.353020 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:17.388044 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:17.413875 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d27a26b8-04ef-4bf7-b759-6f6c15c64acc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86f31b7a33f04f9d8b20794bfca91f10', 'request_id': 'req-d27a26b8-04ef-4bf7-b759-6f6c15c64acc'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:17.448059 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d27a26b8-04ef-4bf7-b759-6f6c15c64acc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86f31b7a33f04f9d8b20794bfca91f10', 'request_id': 'req-d27a26b8-04ef-4bf7-b759-6f6c15c64acc'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:17.460631 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d27a26b8-04ef-4bf7-b759-6f6c15c64acc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86f31b7a33f04f9d8b20794bfca91f10', 'request_id': 'req-d27a26b8-04ef-4bf7-b759-6f6c15c64acc'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:17.480956 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d27a26b8-04ef-4bf7-b759-6f6c15c64acc admin admin] The token cache is being invalidate because project 86f31b7a33f04f9d8b20794bfca91f10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:48:17.481366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d27a26b8-04ef-4bf7-b759-6f6c15c64acc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:17.484395 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d27a26b8-04ef-4bf7-b759-6f6c15c64acc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:17.489474 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 685/1363] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:48:17 2022] DELETE /identity/v3/projects/86f31b7a33f04f9d8b20794bfca91f10 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:17.489474 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2965b2-bee4-4e36-a482-73b4fda7eb3a None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:17.489474 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2965b2-bee4-4e36-a482-73b4fda7eb3a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:17.489474 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2965b2-bee4-4e36-a482-73b4fda7eb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:17.508012 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba5032-d3f2-4679-832d-1f8136b906af admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:17.570323 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-bf2965b2-bee4-4e36-a482-73b4fda7eb3a None None] MFA Rules not processed for user `b882814eba3f4ebcb5e735ea7bf429fb`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:17.573225 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-bf2965b2-bee4-4e36-a482-73b4fda7eb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:17.630088 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25997929-5305-4db4-8cc8-b6d10615a004 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:17.630764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25997929-5305-4db4-8cc8-b6d10615a004 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:17.631050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25997929-5305-4db4-8cc8-b6d10615a004 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:17.631326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25997929-5305-4db4-8cc8-b6d10615a004 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:17.637282 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-25997929-5305-4db4-8cc8-b6d10615a004 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:17.732444 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 686/1364] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:17.738961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52465e68-c143-4c20-ae38-febe906f7c73 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:17.742913 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52465e68-c143-4c20-ae38-febe906f7c73 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:17.743319 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52465e68-c143-4c20-ae38-febe906f7c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:17.782637 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25997929-5305-4db4-8cc8-b6d10615a004 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:17.785985 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25997929-5305-4db4-8cc8-b6d10615a004 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:17.786135 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-52465e68-c143-4c20-ae38-febe906f7c73 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:17.790291 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-52465e68-c143-4c20-ae38-febe906f7c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:17.808503 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 679/1365] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:17.811833 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25997929-5305-4db4-8cc8-b6d10615a004 service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:17.819304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-25997929-5305-4db4-8cc8-b6d10615a004 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:17.848949 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25997929-5305-4db4-8cc8-b6d10615a004 service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:17.864944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf36e55-4df0-482e-8bcc-3f1666905d22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:17.865630 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf36e55-4df0-482e-8bcc-3f1666905d22 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:17.865987 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf36e55-4df0-482e-8bcc-3f1666905d22 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:17.867184 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf36e55-4df0-482e-8bcc-3f1666905d22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:17.894034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf36e55-4df0-482e-8bcc-3f1666905d22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:17.900072 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf36e55-4df0-482e-8bcc-3f1666905d22 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:17.935363 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 680/1366] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:17.968840 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf36e55-4df0-482e-8bcc-3f1666905d22 service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:17.971588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-bcf36e55-4df0-482e-8bcc-3f1666905d22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:17.975153 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 687/1367] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:17.984452 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550833c9-f3c4-474d-a559-05bdf0d8da41 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:17.984859 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550833c9-f3c4-474d-a559-05bdf0d8da41 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:17.985248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550833c9-f3c4-474d-a559-05bdf0d8da41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:18.004352 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf36e55-4df0-482e-8bcc-3f1666905d22 service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:18.023955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05ba4c-d930-4427-9715-c41353538c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:18.026773 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05ba4c-d930-4427-9715-c41353538c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:18.029175 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05ba4c-d930-4427-9715-c41353538c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:18.030642 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05ba4c-d930-4427-9715-c41353538c1a admin admin] PATH_INFO: `/v3/users/b882814eba3f4ebcb5e735ea7bf429fb` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:18.039302 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05ba4c-d930-4427-9715-c41353538c1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b882814eba3f4ebcb5e735ea7bf429fb)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:18.044123 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05ba4c-d930-4427-9715-c41353538c1a admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:18.049302 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-550833c9-f3c4-474d-a559-05bdf0d8da41 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:18.055006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-550833c9-f3c4-474d-a559-05bdf0d8da41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:18.077432 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 688/1368] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:48:18.081527 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550833c9-f3c4-474d-a559-05bdf0d8da41 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:18.103869 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550833c9-f3c4-474d-a559-05bdf0d8da41 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:18.114685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e269913-ef9f-47dd-8fdd-4fe2244bbc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:18.115702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e269913-ef9f-47dd-8fdd-4fe2244bbc93 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:18.122880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e269913-ef9f-47dd-8fdd-4fe2244bbc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:18.124199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e269913-ef9f-47dd-8fdd-4fe2244bbc93 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:18.125657 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e269913-ef9f-47dd-8fdd-4fe2244bbc93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:18.126378 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e269913-ef9f-47dd-8fdd-4fe2244bbc93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:18.129545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e269913-ef9f-47dd-8fdd-4fe2244bbc93 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:18.160766 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 689/1369] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:18.165739 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e269913-ef9f-47dd-8fdd-4fe2244bbc93 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:18.197232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e269913-ef9f-47dd-8fdd-4fe2244bbc93 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:18.203359 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8d05ba4c-d930-4427-9715-c41353538c1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b882814eba3f4ebcb5e735ea7bf429fb'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:18.204347 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8d05ba4c-d930-4427-9715-c41353538c1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b882814eba3f4ebcb5e735ea7bf429fb'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:18.210865 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ceb19b-8061-4ded-adc1-f44a878c7fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:18.214956 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ceb19b-8061-4ded-adc1-f44a878c7fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:18.215479 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ceb19b-8061-4ded-adc1-f44a878c7fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:18.217013 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:18.217256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ceb19b-8061-4ded-adc1-f44a878c7fb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:18.218164 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ceb19b-8061-4ded-adc1-f44a878c7fb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:18.220399 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ceb19b-8061-4ded-adc1-f44a878c7fb1 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:18.233981 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8d05ba4c-d930-4427-9715-c41353538c1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b882814eba3f4ebcb5e735ea7bf429fb'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:18.254907 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8d05ba4c-d930-4427-9715-c41353538c1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b882814eba3f4ebcb5e735ea7bf429fb'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:18.257494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8d05ba4c-d930-4427-9715-c41353538c1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b882814eba3f4ebcb5e735ea7bf429fb'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:18.291382 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 690/1370] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:18 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:18.295147 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ceb19b-8061-4ded-adc1-f44a878c7fb1 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:18.297849 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c3ceb19b-8061-4ded-adc1-f44a878c7fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:18.305886 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 681/1371] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:48:17 2022] DELETE /identity/v3/users/b882814eba3f4ebcb5e735ea7bf429fb => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:18.307982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9856277-a91b-4f5a-b8ac-281ad36c1e93 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:18.308442 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9856277-a91b-4f5a-b8ac-281ad36c1e93 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:18.309085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9856277-a91b-4f5a-b8ac-281ad36c1e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:18.333119 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:18.364916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-b9856277-a91b-4f5a-b8ac-281ad36c1e93 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:18.368731 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b9856277-a91b-4f5a-b8ac-281ad36c1e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:18.524196 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ceb19b-8061-4ded-adc1-f44a878c7fb1 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:18.579541 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 682/1372] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:18.637272 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9856277-a91b-4f5a-b8ac-281ad36c1e93 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:18.639651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b9856277-a91b-4f5a-b8ac-281ad36c1e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:18.673529 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec89b52-408d-4670-aaab-333f1efc78d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:18.674172 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec89b52-408d-4670-aaab-333f1efc78d0 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:18.674455 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec89b52-408d-4670-aaab-333f1efc78d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:18.674729 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec89b52-408d-4670-aaab-333f1efc78d0 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:18.675385 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec89b52-408d-4670-aaab-333f1efc78d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:18.677393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec89b52-408d-4670-aaab-333f1efc78d0 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:18.730131 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 691/1373] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:18 2022] POST /identity/v3/users => generated 286 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:48:18.731753 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec89b52-408d-4670-aaab-333f1efc78d0 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:18.758587 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec89b52-408d-4670-aaab-333f1efc78d0 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:18.774900 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64c4d6-2c66-436f-8abc-5ab5d3c2a14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:18.775536 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64c4d6-2c66-436f-8abc-5ab5d3c2a14f admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:18.775819 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64c4d6-2c66-436f-8abc-5ab5d3c2a14f admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:18.776127 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64c4d6-2c66-436f-8abc-5ab5d3c2a14f admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:18.776821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64c4d6-2c66-436f-8abc-5ab5d3c2a14f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:18.777766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9856277-a91b-4f5a-b8ac-281ad36c1e93 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:18.778443 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64c4d6-2c66-436f-8abc-5ab5d3c2a14f admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:18.782463 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd4688-e1f4-4f8f-900d-fc37625a8609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:18.783099 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd4688-e1f4-4f8f-900d-fc37625a8609 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:18.783549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd4688-e1f4-4f8f-900d-fc37625a8609 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:18.783965 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd4688-e1f4-4f8f-900d-fc37625a8609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:18.795040 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd4688-e1f4-4f8f-900d-fc37625a8609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:18.796818 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd4688-e1f4-4f8f-900d-fc37625a8609 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:18.812066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 683/1374] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:18.833498 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 692/1375] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:18.837588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd4688-e1f4-4f8f-900d-fc37625a8609 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:18.851220 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd4688-e1f4-4f8f-900d-fc37625a8609 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:18.857157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4043cc5-3454-4c3b-87b9-9ecb03363c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:18.858051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4043cc5-3454-4c3b-87b9-9ecb03363c1d admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:18.858900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4043cc5-3454-4c3b-87b9-9ecb03363c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:18.859339 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4043cc5-3454-4c3b-87b9-9ecb03363c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:18.860203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4043cc5-3454-4c3b-87b9-9ecb03363c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:18.863556 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4043cc5-3454-4c3b-87b9-9ecb03363c1d admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:18.882130 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 684/1376] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:48:18.886369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64c4d6-2c66-436f-8abc-5ab5d3c2a14f admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:18.902735 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64c4d6-2c66-436f-8abc-5ab5d3c2a14f admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:18.917640 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521c192-dcd7-478f-b5c0-6276fac36c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:18.918365 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521c192-dcd7-478f-b5c0-6276fac36c95 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:18.918653 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521c192-dcd7-478f-b5c0-6276fac36c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:18.918933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521c192-dcd7-478f-b5c0-6276fac36c95 admin admin] PATH_INFO: `/v3/projects/70c0297f4605423da3ced140c4b4eb58/users/d8b35c3272a64b308183eea33b59ae78/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:18.945888 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521c192-dcd7-478f-b5c0-6276fac36c95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70c0297f4605423da3ced140c4b4eb58, user_id=d8b35c3272a64b308183eea33b59ae78, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:18.949353 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521c192-dcd7-478f-b5c0-6276fac36c95 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:18.969156 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 693/1377] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:18 2022] PUT /identity/v3/projects/70c0297f4605423da3ced140c4b4eb58/users/d8b35c3272a64b308183eea33b59ae78/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:18.980775 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60241eeb-5000-4ef8-be05-5244e5ae8b71 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:18.981253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60241eeb-5000-4ef8-be05-5244e5ae8b71 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:18.981809 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60241eeb-5000-4ef8-be05-5244e5ae8b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:19.019746 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-60241eeb-5000-4ef8-be05-5244e5ae8b71 None None] MFA Rules not processed for user `d8b35c3272a64b308183eea33b59ae78`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:19.021429 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-60241eeb-5000-4ef8-be05-5244e5ae8b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:19.113605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:19.128529 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521c192-dcd7-478f-b5c0-6276fac36c95 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:19.131348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 685/1378] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:19.212752 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b83c0-9fb4-4508-943e-cb7e09d9ca5c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:19.213485 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b83c0-9fb4-4508-943e-cb7e09d9ca5c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:19.213485 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b83c0-9fb4-4508-943e-cb7e09d9ca5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:19.296383 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-4c9b83c0-9fb4-4508-943e-cb7e09d9ca5c None None] MFA Rules not processed for user `df1ee15eb3bc468188a9d8c1af46529c`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:19.301447 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4c9b83c0-9fb4-4508-943e-cb7e09d9ca5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:19.366458 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521c192-dcd7-478f-b5c0-6276fac36c95 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:19.378089 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b62b6-6f49-4af2-a8fd-0ef253c59514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:19.378728 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b62b6-6f49-4af2-a8fd-0ef253c59514 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:19.379042 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b62b6-6f49-4af2-a8fd-0ef253c59514 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:19.379319 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b62b6-6f49-4af2-a8fd-0ef253c59514 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:19.379998 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b62b6-6f49-4af2-a8fd-0ef253c59514 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:19.388758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b62b6-6f49-4af2-a8fd-0ef253c59514 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:19.448310 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:19.466121 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 694/1379] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:19 2022] POST /identity/v3/projects => generated 371 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:19.470876 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b62b6-6f49-4af2-a8fd-0ef253c59514 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:19.508774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 686/1380] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:19.663152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b62b6-6f49-4af2-a8fd-0ef253c59514 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:19.677404 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5314e05-c58e-4ba4-be36-de24f29ff253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:19.677960 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5314e05-c58e-4ba4-be36-de24f29ff253 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:19.678389 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5314e05-c58e-4ba4-be36-de24f29ff253 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:19.678674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5314e05-c58e-4ba4-be36-de24f29ff253 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:19.680362 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5314e05-c58e-4ba4-be36-de24f29ff253 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:19.682319 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5314e05-c58e-4ba4-be36-de24f29ff253 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:19.749202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 695/1381] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:19 2022] POST /identity/v3/users => generated 292 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:19.751177 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b83c0-9fb4-4508-943e-cb7e09d9ca5c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:19.772856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b83c0-9fb4-4508-943e-cb7e09d9ca5c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:19.792845 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a435b4-d29b-49fe-9be8-0910630238a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:19.793493 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a435b4-d29b-49fe-9be8-0910630238a9 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:19.793776 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a435b4-d29b-49fe-9be8-0910630238a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:19.794491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a435b4-d29b-49fe-9be8-0910630238a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:19.795690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a435b4-d29b-49fe-9be8-0910630238a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:19.801393 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a435b4-d29b-49fe-9be8-0910630238a9 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:19.837096 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5314e05-c58e-4ba4-be36-de24f29ff253 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:19.848835 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 687/1382] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:19.860396 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5314e05-c58e-4ba4-be36-de24f29ff253 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:19.885818 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bfde3-8753-4d79-94e4-803522bdc91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:19.889484 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bfde3-8753-4d79-94e4-803522bdc91a admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:19.889841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bfde3-8753-4d79-94e4-803522bdc91a admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:19.890123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bfde3-8753-4d79-94e4-803522bdc91a admin admin] PATH_INFO: `/v3/projects/3ee356ebd4a048bb99322a3219c25020/users/d487fb82bf4344239c8cbd841e39ece3/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:19.937172 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bfde3-8753-4d79-94e4-803522bdc91a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee356ebd4a048bb99322a3219c25020, user_id=d487fb82bf4344239c8cbd841e39ece3, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:19.940767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bfde3-8753-4d79-94e4-803522bdc91a admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:19.970246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a435b4-d29b-49fe-9be8-0910630238a9 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:19.970693 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 696/1383] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:19 2022] PUT /identity/v3/projects/3ee356ebd4a048bb99322a3219c25020/users/d487fb82bf4344239c8cbd841e39ece3/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:19.984640 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a435b4-d29b-49fe-9be8-0910630238a9 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:20.099403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bfde3-8753-4d79-94e4-803522bdc91a admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:20.101243 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-344bfde3-8753-4d79-94e4-803522bdc91a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:20.155232 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70367e2a-1d8f-4f9c-b49c-992c065de491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:20.156200 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70367e2a-1d8f-4f9c-b49c-992c065de491 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.157848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70367e2a-1d8f-4f9c-b49c-992c065de491 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.159972 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70367e2a-1d8f-4f9c-b49c-992c065de491 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:20.161074 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70367e2a-1d8f-4f9c-b49c-992c065de491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:20.166882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70367e2a-1d8f-4f9c-b49c-992c065de491 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:20.197753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 688/1384] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:20.200629 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70367e2a-1d8f-4f9c-b49c-992c065de491 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:20.229244 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70367e2a-1d8f-4f9c-b49c-992c065de491 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:20.242476 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2514b-bdbe-4c14-a5bf-93a5163bd202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:20.243246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2514b-bdbe-4c14-a5bf-93a5163bd202 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.245662 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2514b-bdbe-4c14-a5bf-93a5163bd202 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.246125 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2514b-bdbe-4c14-a5bf-93a5163bd202 admin admin] PATH_INFO: `/v3/projects/3ee356ebd4a048bb99322a3219c25020/users/d487fb82bf4344239c8cbd841e39ece3/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:20.267771 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2514b-bdbe-4c14-a5bf-93a5163bd202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee356ebd4a048bb99322a3219c25020, user_id=d487fb82bf4344239c8cbd841e39ece3, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:20.273648 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2514b-bdbe-4c14-a5bf-93a5163bd202 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:20.290119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 689/1385] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:20 2022] PUT /identity/v3/projects/3ee356ebd4a048bb99322a3219c25020/users/d487fb82bf4344239c8cbd841e39ece3/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:20.303193 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039195d-3b90-4a73-a924-676793d4ce86 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.303522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039195d-3b90-4a73-a924-676793d4ce86 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.303758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039195d-3b90-4a73-a924-676793d4ce86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:20.306197 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bfde3-8753-4d79-94e4-803522bdc91a admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:20.373606 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-e039195d-3b90-4a73-a924-676793d4ce86 None None] MFA Rules not processed for user `d487fb82bf4344239c8cbd841e39ece3`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:20.374962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e039195d-3b90-4a73-a924-676793d4ce86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:20.398799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ba27f-cb71-4a6b-8539-721bf0965aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:20.401135 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ba27f-cb71-4a6b-8539-721bf0965aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.401535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ba27f-cb71-4a6b-8539-721bf0965aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.401915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ba27f-cb71-4a6b-8539-721bf0965aa9 admin admin] PATH_INFO: `/v3/users/258b8f46bd88425e89da0d81cbdb681b` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:20.407752 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ba27f-cb71-4a6b-8539-721bf0965aa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=258b8f46bd88425e89da0d81cbdb681b)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:20.412142 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ba27f-cb71-4a6b-8539-721bf0965aa9 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:20.437874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:20.528755 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-676ba27f-cb71-4a6b-8539-721bf0965aa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '258b8f46bd88425e89da0d81cbdb681b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:20.530717 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-676ba27f-cb71-4a6b-8539-721bf0965aa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '258b8f46bd88425e89da0d81cbdb681b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:20.559270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-676ba27f-cb71-4a6b-8539-721bf0965aa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '258b8f46bd88425e89da0d81cbdb681b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:20.566359 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:20.569410 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-676ba27f-cb71-4a6b-8539-721bf0965aa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '258b8f46bd88425e89da0d81cbdb681b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:20.574945 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 690/1386] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:20 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:20.577448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039195d-3b90-4a73-a924-676793d4ce86 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:20.582653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e039195d-3b90-4a73-a924-676793d4ce86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:20.589746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-676ba27f-cb71-4a6b-8539-721bf0965aa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '258b8f46bd88425e89da0d81cbdb681b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:20.604172 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 697/1387] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:48:20 2022] DELETE /identity/v3/users/258b8f46bd88425e89da0d81cbdb681b => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:20.605610 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ba27f-cb71-4a6b-8539-721bf0965aa9 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:20.606855 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-676ba27f-cb71-4a6b-8539-721bf0965aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:20.621308 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039195d-3b90-4a73-a924-676793d4ce86 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:20.700570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ba27f-cb71-4a6b-8539-721bf0965aa9 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:20.710386 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e22b1-19c1-46cb-af13-882f55f77ec5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:20.714951 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e22b1-19c1-46cb-af13-882f55f77ec5 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.715091 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e22b1-19c1-46cb-af13-882f55f77ec5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.715388 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e22b1-19c1-46cb-af13-882f55f77ec5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:20.752628 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810b9be-7b0b-44b5-a8c5-94383ffb158a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:20.753294 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810b9be-7b0b-44b5-a8c5-94383ffb158a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.753581 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810b9be-7b0b-44b5-a8c5-94383ffb158a admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.753858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810b9be-7b0b-44b5-a8c5-94383ffb158a admin admin] PATH_INFO: `/v3/projects/1e439515385741ca90d0429882c1f84b` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:20.755402 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810b9be-7b0b-44b5-a8c5-94383ffb158a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e439515385741ca90d0429882c1f84b)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:20.757538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810b9be-7b0b-44b5-a8c5-94383ffb158a admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:20.844745 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e22b1-19c1-46cb-af13-882f55f77ec5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:20.846284 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e22b1-19c1-46cb-af13-882f55f77ec5 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:20.859817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2810b9be-7b0b-44b5-a8c5-94383ffb158a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e439515385741ca90d0429882c1f84b', 'request_id': 'req-2810b9be-7b0b-44b5-a8c5-94383ffb158a'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:20.864344 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 698/1388] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:20.865752 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977814c8-3d4f-42e3-9581-9417b9524183 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.866051 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977814c8-3d4f-42e3-9581-9417b9524183 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.867030 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977814c8-3d4f-42e3-9581-9417b9524183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:20.893740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2810b9be-7b0b-44b5-a8c5-94383ffb158a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e439515385741ca90d0429882c1f84b', 'request_id': 'req-2810b9be-7b0b-44b5-a8c5-94383ffb158a'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:20.905160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2810b9be-7b0b-44b5-a8c5-94383ffb158a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e439515385741ca90d0429882c1f84b', 'request_id': 'req-2810b9be-7b0b-44b5-a8c5-94383ffb158a'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:20.922736 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-977814c8-3d4f-42e3-9581-9417b9524183 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:20.924007 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-977814c8-3d4f-42e3-9581-9417b9524183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:20.928364 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2810b9be-7b0b-44b5-a8c5-94383ffb158a admin admin] The token cache is being invalidate because project 1e439515385741ca90d0429882c1f84b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:48:20.928906 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2810b9be-7b0b-44b5-a8c5-94383ffb158a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:20.946333 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2810b9be-7b0b-44b5-a8c5-94383ffb158a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:20.950399 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 691/1389] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:48:20 2022] DELETE /identity/v3/projects/1e439515385741ca90d0429882c1f84b => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:20.952643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c15f2d-0a1d-4768-89a5-b3a3c80638c5 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.952934 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c15f2d-0a1d-4768-89a5-b3a3c80638c5 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.953212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c15f2d-0a1d-4768-89a5-b3a3c80638c5 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:20.957081 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 692/1390] 10.222.0.36 () {54 vars in 912 bytes} [Sun Oct 30 00:48:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:20.958322 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d9188-22b4-4173-bdab-047f391f3334 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:20.958610 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d9188-22b4-4173-bdab-047f391f3334 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:20.958888 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d9188-22b4-4173-bdab-047f391f3334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:21.021711 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:21.023733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-b56d9188-22b4-4173-bdab-047f391f3334 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:21.024192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b56d9188-22b4-4173-bdab-047f391f3334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:21.081397 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 699/1391] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:21.082711 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8018d-0700-4d6e-8f7d-0a6f6a07679d None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:21.082988 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8018d-0700-4d6e-8f7d-0a6f6a07679d None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:21.083269 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8018d-0700-4d6e-8f7d-0a6f6a07679d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:21.168009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-12b8018d-0700-4d6e-8f7d-0a6f6a07679d None None] MFA Rules not processed for user `fb39b6cc97a1456bb1312f49118c6e10`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:21.169946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-12b8018d-0700-4d6e-8f7d-0a6f6a07679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:21.245323 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 700/1392] 10.222.0.36 () {58 vars in 1009 bytes} [Sun Oct 30 00:48:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:21.246607 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b8018d-0700-4d6e-8f7d-0a6f6a07679d None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:21.248110 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-12b8018d-0700-4d6e-8f7d-0a6f6a07679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:21.262333 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b8018d-0700-4d6e-8f7d-0a6f6a07679d None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:21.267209 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237103f-608f-4442-8e0c-edd237179913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:21.267977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237103f-608f-4442-8e0c-edd237179913 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:21.268319 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237103f-608f-4442-8e0c-edd237179913 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:21.268599 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237103f-608f-4442-8e0c-edd237179913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:21.270151 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 693/1393] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:21.283201 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064684a-f2cb-4a0e-a2bd-66c65d34e2c2 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:21.283937 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237103f-608f-4442-8e0c-edd237179913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:21.286475 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064684a-f2cb-4a0e-a2bd-66c65d34e2c2 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:21.288185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237103f-608f-4442-8e0c-edd237179913 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:21.289893 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064684a-f2cb-4a0e-a2bd-66c65d34e2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:21.305287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 701/1394] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:21.307097 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237103f-608f-4442-8e0c-edd237179913 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:21.323734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237103f-608f-4442-8e0c-edd237179913 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:21.329633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917dd0a-d96d-4588-9776-60ca43fcb0a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:21.330588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917dd0a-d96d-4588-9776-60ca43fcb0a5 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:21.330888 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917dd0a-d96d-4588-9776-60ca43fcb0a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:21.331968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917dd0a-d96d-4588-9776-60ca43fcb0a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:21.349521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917dd0a-d96d-4588-9776-60ca43fcb0a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:21.351662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917dd0a-d96d-4588-9776-60ca43fcb0a5 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:21.357320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-9064684a-f2cb-4a0e-a2bd-66c65d34e2c2 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:21.360356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-9064684a-f2cb-4a0e-a2bd-66c65d34e2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:21.372912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 702/1395] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:21.392343 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 694/1396] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:21.393717 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917dd0a-d96d-4588-9776-60ca43fcb0a5 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:21.415469 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917dd0a-d96d-4588-9776-60ca43fcb0a5 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:21.426857 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc9d18-0c44-45f8-8333-d5d95fd1c898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:21.427679 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc9d18-0c44-45f8-8333-d5d95fd1c898 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:21.428144 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc9d18-0c44-45f8-8333-d5d95fd1c898 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:21.428593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc9d18-0c44-45f8-8333-d5d95fd1c898 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:21.429419 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc9d18-0c44-45f8-8333-d5d95fd1c898 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:21.429929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc9d18-0c44-45f8-8333-d5d95fd1c898 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:21.432004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc9d18-0c44-45f8-8333-d5d95fd1c898 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:21.465915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 703/1397] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:21.468143 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064684a-f2cb-4a0e-a2bd-66c65d34e2c2 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:21.478753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064684a-f2cb-4a0e-a2bd-66c65d34e2c2 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:21.487109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f5d0a-9058-4f3e-90dc-a1aa41d9a096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:21.487823 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f5d0a-9058-4f3e-90dc-a1aa41d9a096 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:21.488586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f5d0a-9058-4f3e-90dc-a1aa41d9a096 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:21.488586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f5d0a-9058-4f3e-90dc-a1aa41d9a096 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:21.489275 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f5d0a-9058-4f3e-90dc-a1aa41d9a096 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:21.491439 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f5d0a-9058-4f3e-90dc-a1aa41d9a096 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:21.537347 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 695/1398] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:21 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:21.544327 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc9d18-0c44-45f8-8333-d5d95fd1c898 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:21.562312 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc9d18-0c44-45f8-8333-d5d95fd1c898 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:21.777504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f5d0a-9058-4f3e-90dc-a1aa41d9a096 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:21.782991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-986f5d0a-9058-4f3e-90dc-a1aa41d9a096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:21.790318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:21.841796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da1a38-0fba-4fb3-84a4-bf083d390b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:21.845027 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da1a38-0fba-4fb3-84a4-bf083d390b57 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:21.845317 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da1a38-0fba-4fb3-84a4-bf083d390b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:21.845706 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da1a38-0fba-4fb3-84a4-bf083d390b57 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:21.846916 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da1a38-0fba-4fb3-84a4-bf083d390b57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:21.849635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da1a38-0fba-4fb3-84a4-bf083d390b57 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:21.891212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 704/1399] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:21 2022] POST /identity/v3/users => generated 289 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:21.895601 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da1a38-0fba-4fb3-84a4-bf083d390b57 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:21.917204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da1a38-0fba-4fb3-84a4-bf083d390b57 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:21.952868 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8d7fe-f9ac-4f63-96e5-6d50401ef938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:21.954617 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8d7fe-f9ac-4f63-96e5-6d50401ef938 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:21.954930 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8d7fe-f9ac-4f63-96e5-6d50401ef938 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:21.955270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8d7fe-f9ac-4f63-96e5-6d50401ef938 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:21.956894 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8d7fe-f9ac-4f63-96e5-6d50401ef938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:21.959598 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8d7fe-f9ac-4f63-96e5-6d50401ef938 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:21.979962 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 705/1400] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:21.986118 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8d7fe-f9ac-4f63-96e5-6d50401ef938 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:22.026102 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8d7fe-f9ac-4f63-96e5-6d50401ef938 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:22.036775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8edf9-c543-46ef-84f2-a8d090d0f2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:22.037751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8edf9-c543-46ef-84f2-a8d090d0f2dd admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:22.038144 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8edf9-c543-46ef-84f2-a8d090d0f2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:22.038519 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8edf9-c543-46ef-84f2-a8d090d0f2dd admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:22.039265 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8edf9-c543-46ef-84f2-a8d090d0f2dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:22.041935 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8edf9-c543-46ef-84f2-a8d090d0f2dd admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:22.054162 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f5d0a-9058-4f3e-90dc-a1aa41d9a096 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:22.079599 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133b1b8-2fc1-409a-9a00-195634ce6891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:22.080302 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133b1b8-2fc1-409a-9a00-195634ce6891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:22.080593 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133b1b8-2fc1-409a-9a00-195634ce6891 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:22.080949 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133b1b8-2fc1-409a-9a00-195634ce6891 admin admin] PATH_INFO: `/v3/users/d8b35c3272a64b308183eea33b59ae78` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:22.084429 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 706/1401] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:22.084790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133b1b8-2fc1-409a-9a00-195634ce6891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8b35c3272a64b308183eea33b59ae78)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:22.086788 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133b1b8-2fc1-409a-9a00-195634ce6891 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:22.094413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8edf9-c543-46ef-84f2-a8d090d0f2dd admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:22.109809 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8edf9-c543-46ef-84f2-a8d090d0f2dd admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:22.119907 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9246511-f33e-410f-a600-6969b8a0adec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:22.120634 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9246511-f33e-410f-a600-6969b8a0adec admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:22.120917 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9246511-f33e-410f-a600-6969b8a0adec admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:22.121224 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9246511-f33e-410f-a600-6969b8a0adec admin admin] PATH_INFO: `/v3/projects/240fafb59a9446c784beae25f92c532d/users/6b20903295fe4c638a2e7656a4ba74e1/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:22.134322 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:22.160625 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9246511-f33e-410f-a600-6969b8a0adec admin admin] RBAC: Authorizing `identity:create_grant(project_id=240fafb59a9446c784beae25f92c532d, user_id=6b20903295fe4c638a2e7656a4ba74e1, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:22.164601 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9246511-f33e-410f-a600-6969b8a0adec admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:22.197835 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-4133b1b8-2fc1-409a-9a00-195634ce6891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8b35c3272a64b308183eea33b59ae78'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:22.198530 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-4133b1b8-2fc1-409a-9a00-195634ce6891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8b35c3272a64b308183eea33b59ae78'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:22.216059 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 707/1402] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:22 2022] PUT /identity/v3/projects/240fafb59a9446c784beae25f92c532d/users/6b20903295fe4c638a2e7656a4ba74e1/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:22.228660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-4133b1b8-2fc1-409a-9a00-195634ce6891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8b35c3272a64b308183eea33b59ae78'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:22.232680 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d35cf-b8dd-4c15-94af-d404a5252345 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:22.232994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d35cf-b8dd-4c15-94af-d404a5252345 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:22.233256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d35cf-b8dd-4c15-94af-d404a5252345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:22.244656 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-4133b1b8-2fc1-409a-9a00-195634ce6891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8b35c3272a64b308183eea33b59ae78'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:22.245738 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-4133b1b8-2fc1-409a-9a00-195634ce6891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8b35c3272a64b308183eea33b59ae78'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:22.270148 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 696/1403] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:48:21 2022] DELETE /identity/v3/users/d8b35c3272a64b308183eea33b59ae78 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:22.279615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f8433-0138-4e1c-ad6b-e7cc093d651e None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:22.279857 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f8433-0138-4e1c-ad6b-e7cc093d651e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:22.280119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f8433-0138-4e1c-ad6b-e7cc093d651e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:22.282398 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-7b5d35cf-b8dd-4c15-94af-d404a5252345 None None] MFA Rules not processed for user `6b20903295fe4c638a2e7656a4ba74e1`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:22.283715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-7b5d35cf-b8dd-4c15-94af-d404a5252345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:22.329627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-8a5f8433-0138-4e1c-ad6b-e7cc093d651e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:22.331533 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-8a5f8433-0138-4e1c-ad6b-e7cc093d651e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:22.454363 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 708/1404] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:22.456237 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d35cf-b8dd-4c15-94af-d404a5252345 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:22.472288 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-7b5d35cf-b8dd-4c15-94af-d404a5252345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:22.505682 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 697/1405] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:22.520681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d35cf-b8dd-4c15-94af-d404a5252345 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:22.538891 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171b742-ae06-4efa-9a83-23286e9e663a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:22.539862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171b742-ae06-4efa-9a83-23286e9e663a admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:22.540003 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171b742-ae06-4efa-9a83-23286e9e663a admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:22.540220 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171b742-ae06-4efa-9a83-23286e9e663a admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:22.540869 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171b742-ae06-4efa-9a83-23286e9e663a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:22.541466 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f8433-0138-4e1c-ad6b-e7cc093d651e None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:22.543098 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-8a5f8433-0138-4e1c-ad6b-e7cc093d651e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:22.543222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171b742-ae06-4efa-9a83-23286e9e663a admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:22.621632 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 709/1406] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:22 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:22.622717 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171b742-ae06-4efa-9a83-23286e9e663a admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:22.629002 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:22.729349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f8433-0138-4e1c-ad6b-e7cc093d651e None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:22.798340 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171b742-ae06-4efa-9a83-23286e9e663a admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:22.808008 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77566b19-a8dd-44e8-bdd8-13e701802897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:22.808856 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77566b19-a8dd-44e8-bdd8-13e701802897 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:22.808973 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77566b19-a8dd-44e8-bdd8-13e701802897 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:22.809245 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77566b19-a8dd-44e8-bdd8-13e701802897 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:22.809911 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77566b19-a8dd-44e8-bdd8-13e701802897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:22.814951 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77566b19-a8dd-44e8-bdd8-13e701802897 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:22.872799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 710/1407] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:22 2022] POST /identity/v3/users => generated 289 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:22.874229 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77566b19-a8dd-44e8-bdd8-13e701802897 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:22.887137 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1289ebc-c7f7-49eb-b8f4-ead70a5553cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:22.887785 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1289ebc-c7f7-49eb-b8f4-ead70a5553cd service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:22.888105 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1289ebc-c7f7-49eb-b8f4-ead70a5553cd service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:22.888379 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1289ebc-c7f7-49eb-b8f4-ead70a5553cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:22.897502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77566b19-a8dd-44e8-bdd8-13e701802897 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:22.913157 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004adab-a8ea-4151-b698-870b208035e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:22.913803 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004adab-a8ea-4151-b698-870b208035e3 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:22.914085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004adab-a8ea-4151-b698-870b208035e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:22.914358 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004adab-a8ea-4151-b698-870b208035e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:22.916060 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5004adab-a8ea-4151-b698-870b208035e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:22.918315 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5004adab-a8ea-4151-b698-870b208035e3 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:22.919649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1289ebc-c7f7-49eb-b8f4-ead70a5553cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:22.921816 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1289ebc-c7f7-49eb-b8f4-ead70a5553cd service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:22.948646 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 698/1408] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:22.952808 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 711/1409] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:22.953660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1289ebc-c7f7-49eb-b8f4-ead70a5553cd service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:22.981046 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1289ebc-c7f7-49eb-b8f4-ead70a5553cd service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:22.992983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2de0d6-9122-4240-84d9-abb57abfd6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:22.993780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2de0d6-9122-4240-84d9-abb57abfd6f2 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:22.993960 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2de0d6-9122-4240-84d9-abb57abfd6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:22.994181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2de0d6-9122-4240-84d9-abb57abfd6f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:22.994884 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2de0d6-9122-4240-84d9-abb57abfd6f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:22.996664 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2de0d6-9122-4240-84d9-abb57abfd6f2 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:23.053387 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 699/1410] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:23.055110 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004adab-a8ea-4151-b698-870b208035e3 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:23.086700 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004adab-a8ea-4151-b698-870b208035e3 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:23.103842 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63887baf-8791-40a7-bfb9-8540b496fac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:23.116310 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63887baf-8791-40a7-bfb9-8540b496fac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:23.116597 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63887baf-8791-40a7-bfb9-8540b496fac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:23.116982 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63887baf-8791-40a7-bfb9-8540b496fac7 admin admin] PATH_INFO: `/v3/projects/233b15d12d694eb88b15f9c30a61e105/users/4cfb0decbe404e19a98555ca5b9cf636/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:23.151037 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63887baf-8791-40a7-bfb9-8540b496fac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=233b15d12d694eb88b15f9c30a61e105, user_id=4cfb0decbe404e19a98555ca5b9cf636, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:23.155422 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63887baf-8791-40a7-bfb9-8540b496fac7 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:23.186553 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 712/1411] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:23 2022] PUT /identity/v3/projects/233b15d12d694eb88b15f9c30a61e105/users/4cfb0decbe404e19a98555ca5b9cf636/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:23.189847 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c09dcf-e8a7-4e92-9c9b-33c44e25094f None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:23.190165 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c09dcf-e8a7-4e92-9c9b-33c44e25094f None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:23.190448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c09dcf-e8a7-4e92-9c9b-33c44e25094f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:23.302810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-91c09dcf-e8a7-4e92-9c9b-33c44e25094f None None] MFA Rules not processed for user `4cfb0decbe404e19a98555ca5b9cf636`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:23.304099 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-91c09dcf-e8a7-4e92-9c9b-33c44e25094f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:23.491037 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 700/1412] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:23.498663 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63887baf-8791-40a7-bfb9-8540b496fac7 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:23.509642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:23.590572 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c09dcf-e8a7-4e92-9c9b-33c44e25094f None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:23.593340 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-91c09dcf-e8a7-4e92-9c9b-33c44e25094f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:23.623940 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63887baf-8791-40a7-bfb9-8540b496fac7 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:23.638279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becf4e5-1fe7-4484-8227-ae497272861d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:23.640053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becf4e5-1fe7-4484-8227-ae497272861d admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:23.640640 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becf4e5-1fe7-4484-8227-ae497272861d admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:23.642133 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becf4e5-1fe7-4484-8227-ae497272861d admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:23.643229 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2becf4e5-1fe7-4484-8227-ae497272861d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:23.648167 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2becf4e5-1fe7-4484-8227-ae497272861d admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:23.726200 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 713/1413] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:23 2022] POST /identity/v3/projects => generated 375 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:23.729820 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becf4e5-1fe7-4484-8227-ae497272861d admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:23.736599 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-2becf4e5-1fe7-4484-8227-ae497272861d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:23.802115 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:23.831656 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c09dcf-e8a7-4e92-9c9b-33c44e25094f None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:24.036283 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8576ea8-29cf-4838-a1bc-d46f64291c4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:24.039429 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8576ea8-29cf-4838-a1bc-d46f64291c4f service placement] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:24.040884 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8576ea8-29cf-4838-a1bc-d46f64291c4f service placement] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:24.041887 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8576ea8-29cf-4838-a1bc-d46f64291c4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:24.051902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-f8576ea8-29cf-4838-a1bc-d46f64291c4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:24.133196 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8576ea8-29cf-4838-a1bc-d46f64291c4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:24.142824 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8576ea8-29cf-4838-a1bc-d46f64291c4f service placement] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:24.212484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 701/1414] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:48:24.214089 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8576ea8-29cf-4838-a1bc-d46f64291c4f service placement] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:24.256804 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becf4e5-1fe7-4484-8227-ae497272861d admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:24.272544 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2663b2-08b8-4de6-92d2-2cb8c2576894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:24.273389 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2663b2-08b8-4de6-92d2-2cb8c2576894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:24.273819 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2663b2-08b8-4de6-92d2-2cb8c2576894 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:24.274345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2663b2-08b8-4de6-92d2-2cb8c2576894 admin admin] PATH_INFO: `/v3/projects/4fecc8a40c97490ebb78edb763c2e4d4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:24.276016 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2663b2-08b8-4de6-92d2-2cb8c2576894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fecc8a40c97490ebb78edb763c2e4d4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:24.280745 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2663b2-08b8-4de6-92d2-2cb8c2576894 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:24.382679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8576ea8-29cf-4838-a1bc-d46f64291c4f service placement] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:24.385490 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3a2663b2-08b8-4de6-92d2-2cb8c2576894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fecc8a40c97490ebb78edb763c2e4d4', 'request_id': 'req-3a2663b2-08b8-4de6-92d2-2cb8c2576894'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:24.408917 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3a2663b2-08b8-4de6-92d2-2cb8c2576894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fecc8a40c97490ebb78edb763c2e4d4', 'request_id': 'req-3a2663b2-08b8-4de6-92d2-2cb8c2576894'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:24.425165 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3a2663b2-08b8-4de6-92d2-2cb8c2576894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fecc8a40c97490ebb78edb763c2e4d4', 'request_id': 'req-3a2663b2-08b8-4de6-92d2-2cb8c2576894'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:24.465436 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3a2663b2-08b8-4de6-92d2-2cb8c2576894 admin admin] The token cache is being invalidate because project 4fecc8a40c97490ebb78edb763c2e4d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:48:24.465808 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3a2663b2-08b8-4de6-92d2-2cb8c2576894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:24.467453 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3a2663b2-08b8-4de6-92d2-2cb8c2576894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:24.475060 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 714/1415] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:48:23 2022] DELETE /identity/v3/projects/4fecc8a40c97490ebb78edb763c2e4d4 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:24.476925 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cef501-272c-4334-8cae-f8b06b934fdd None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:24.477386 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cef501-272c-4334-8cae-f8b06b934fdd None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:24.477762 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cef501-272c-4334-8cae-f8b06b934fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:24.552884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-35cef501-272c-4334-8cae-f8b06b934fdd None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:24.554993 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-35cef501-272c-4334-8cae-f8b06b934fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:24.616783 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:24.636905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc73f6-7131-4a66-9b83-7563706059af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:24.637670 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc73f6-7131-4a66-9b83-7563706059af admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:24.638168 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc73f6-7131-4a66-9b83-7563706059af admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:24.638486 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc73f6-7131-4a66-9b83-7563706059af admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:24.639409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc73f6-7131-4a66-9b83-7563706059af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:24.642583 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc73f6-7131-4a66-9b83-7563706059af admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:24.720469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 702/1416] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:24 2022] POST /identity/v3/users => generated 294 bytes in 507 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:24.724261 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc73f6-7131-4a66-9b83-7563706059af admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:24.733551 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-cfcc73f6-7131-4a66-9b83-7563706059af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:24.765567 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc73f6-7131-4a66-9b83-7563706059af admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:24.774704 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 715/1417] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:24.776642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60adef6-37b6-4321-b13c-6ba9c483e219 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:24.777549 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60adef6-37b6-4321-b13c-6ba9c483e219 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:24.777549 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60adef6-37b6-4321-b13c-6ba9c483e219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:24.790245 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2db216-3d04-4dac-9e6c-2c4b48c087ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:24.790792 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2db216-3d04-4dac-9e6c-2c4b48c087ff admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:24.791080 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2db216-3d04-4dac-9e6c-2c4b48c087ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:24.791659 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2db216-3d04-4dac-9e6c-2c4b48c087ff admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:24.792329 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2db216-3d04-4dac-9e6c-2c4b48c087ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:24.793838 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2db216-3d04-4dac-9e6c-2c4b48c087ff admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:24.831640 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 703/1418] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:24.832688 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2db216-3d04-4dac-9e6c-2c4b48c087ff admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:24.855633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2db216-3d04-4dac-9e6c-2c4b48c087ff admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:24.864874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-a60adef6-37b6-4321-b13c-6ba9c483e219 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:24.870840 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f63d5-414a-4459-af8b-3e0ca2b7c599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:24.872030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f63d5-414a-4459-af8b-3e0ca2b7c599 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:24.872493 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f63d5-414a-4459-af8b-3e0ca2b7c599 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:24.872929 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f63d5-414a-4459-af8b-3e0ca2b7c599 admin admin] PATH_INFO: `/v3/projects/bf980aeaf59b4c96ab489c41cdbebe25/users/cb0d97d08f9746de97f2faa0281e5696/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:24.881167 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a60adef6-37b6-4321-b13c-6ba9c483e219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:24.918767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 716/1419] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:24.919744 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60adef6-37b6-4321-b13c-6ba9c483e219 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:24.926269 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a60adef6-37b6-4321-b13c-6ba9c483e219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:24.941718 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60adef6-37b6-4321-b13c-6ba9c483e219 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:24.941944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f63d5-414a-4459-af8b-3e0ca2b7c599 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf980aeaf59b4c96ab489c41cdbebe25, user_id=cb0d97d08f9746de97f2faa0281e5696, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:24.944681 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f63d5-414a-4459-af8b-3e0ca2b7c599 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:24.948067 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:24.949345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe8986-7357-4f3a-8bd2-6c39b955f3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:24.950177 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe8986-7357-4f3a-8bd2-6c39b955f3bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:24.950456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe8986-7357-4f3a-8bd2-6c39b955f3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:24.950730 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe8986-7357-4f3a-8bd2-6c39b955f3bf admin admin] PATH_INFO: `/v3/projects/70c0297f4605423da3ced140c4b4eb58` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:24.954697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe8986-7357-4f3a-8bd2-6c39b955f3bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=70c0297f4605423da3ced140c4b4eb58)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:24.957204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe8986-7357-4f3a-8bd2-6c39b955f3bf admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:24.986297 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 704/1420] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:24 2022] PUT /identity/v3/projects/bf980aeaf59b4c96ab489c41cdbebe25/users/cb0d97d08f9746de97f2faa0281e5696/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:24.989530 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f63d5-414a-4459-af8b-3e0ca2b7c599 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:25.059796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-cabe8986-7357-4f3a-8bd2-6c39b955f3bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70c0297f4605423da3ced140c4b4eb58', 'request_id': 'req-cabe8986-7357-4f3a-8bd2-6c39b955f3bf'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:25.091582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-cabe8986-7357-4f3a-8bd2-6c39b955f3bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70c0297f4605423da3ced140c4b4eb58', 'request_id': 'req-cabe8986-7357-4f3a-8bd2-6c39b955f3bf'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:25.107184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-cabe8986-7357-4f3a-8bd2-6c39b955f3bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70c0297f4605423da3ced140c4b4eb58', 'request_id': 'req-cabe8986-7357-4f3a-8bd2-6c39b955f3bf'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:25.146100 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-cabe8986-7357-4f3a-8bd2-6c39b955f3bf admin admin] The token cache is being invalidate because project 70c0297f4605423da3ced140c4b4eb58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:48:25.146408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-cabe8986-7357-4f3a-8bd2-6c39b955f3bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:25.152457 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-cabe8986-7357-4f3a-8bd2-6c39b955f3bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:25.168525 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 717/1421] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:48:24 2022] DELETE /identity/v3/projects/70c0297f4605423da3ced140c4b4eb58 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:25.169627 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe8986-7357-4f3a-8bd2-6c39b955f3bf admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:25.171547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-cabe8986-7357-4f3a-8bd2-6c39b955f3bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:25.258528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f63d5-414a-4459-af8b-3e0ca2b7c599 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:25.404325 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe8986-7357-4f3a-8bd2-6c39b955f3bf admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:25.425634 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f650735-b3b6-404f-95c9-c9a28bb1a6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:25.426609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f650735-b3b6-404f-95c9-c9a28bb1a6ee admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:25.427091 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f650735-b3b6-404f-95c9-c9a28bb1a6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:25.427550 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f650735-b3b6-404f-95c9-c9a28bb1a6ee admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:25.428439 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f650735-b3b6-404f-95c9-c9a28bb1a6ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:25.430485 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f650735-b3b6-404f-95c9-c9a28bb1a6ee admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:25.456500 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 718/1422] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:25.468028 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f650735-b3b6-404f-95c9-c9a28bb1a6ee admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:25.482980 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-8f650735-b3b6-404f-95c9-c9a28bb1a6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:25.503832 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f63b3a-4d4f-4296-91c0-bce9a2c0a445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:25.505166 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f63b3a-4d4f-4296-91c0-bce9a2c0a445 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:25.505644 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f63b3a-4d4f-4296-91c0-bce9a2c0a445 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:25.506093 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f63b3a-4d4f-4296-91c0-bce9a2c0a445 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:25.506943 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f63b3a-4d4f-4296-91c0-bce9a2c0a445 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:25.507423 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f63b3a-4d4f-4296-91c0-bce9a2c0a445 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:25.509483 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f63b3a-4d4f-4296-91c0-bce9a2c0a445 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:25.531415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f650735-b3b6-404f-95c9-c9a28bb1a6ee admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:25.551148 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31d4e3-7854-4e3f-991f-6a01a3451bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:25.557354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31d4e3-7854-4e3f-991f-6a01a3451bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:25.557921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31d4e3-7854-4e3f-991f-6a01a3451bff admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:25.557921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31d4e3-7854-4e3f-991f-6a01a3451bff admin admin] PATH_INFO: `/v3/users/d487fb82bf4344239c8cbd841e39ece3` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:25.562962 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31d4e3-7854-4e3f-991f-6a01a3451bff admin admin] RBAC: Authorizing `identity:delete_user(user_id=d487fb82bf4344239c8cbd841e39ece3)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:25.564469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 705/1423] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 577 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:25.566768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f63b3a-4d4f-4296-91c0-bce9a2c0a445 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:25.579104 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31d4e3-7854-4e3f-991f-6a01a3451bff admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:25.602630 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f63b3a-4d4f-4296-91c0-bce9a2c0a445 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:25.616886 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2481ea6-6bd1-4323-89cc-5b35bd30ffb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:25.618412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2481ea6-6bd1-4323-89cc-5b35bd30ffb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:25.619668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2481ea6-6bd1-4323-89cc-5b35bd30ffb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:25.620360 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2481ea6-6bd1-4323-89cc-5b35bd30ffb6 admin admin] PATH_INFO: `/v3/projects/bf980aeaf59b4c96ab489c41cdbebe25/users/cb0d97d08f9746de97f2faa0281e5696/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:25.627059 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2481ea6-6bd1-4323-89cc-5b35bd30ffb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf980aeaf59b4c96ab489c41cdbebe25, user_id=cb0d97d08f9746de97f2faa0281e5696, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:25.636677 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2481ea6-6bd1-4323-89cc-5b35bd30ffb6 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:25.673582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 706/1424] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:25 2022] PUT /identity/v3/projects/bf980aeaf59b4c96ab489c41cdbebe25/users/cb0d97d08f9746de97f2faa0281e5696/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:25.676394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:25.691712 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2481ea6-6bd1-4323-89cc-5b35bd30ffb6 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:25.695991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e2481ea6-6bd1-4323-89cc-5b35bd30ffb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:25.715760 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-dc31d4e3-7854-4e3f-991f-6a01a3451bff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd487fb82bf4344239c8cbd841e39ece3'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:25.719180 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-dc31d4e3-7854-4e3f-991f-6a01a3451bff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd487fb82bf4344239c8cbd841e39ece3'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:25.750810 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-dc31d4e3-7854-4e3f-991f-6a01a3451bff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd487fb82bf4344239c8cbd841e39ece3'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:25.771075 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-dc31d4e3-7854-4e3f-991f-6a01a3451bff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd487fb82bf4344239c8cbd841e39ece3'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:25.785454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-dc31d4e3-7854-4e3f-991f-6a01a3451bff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd487fb82bf4344239c8cbd841e39ece3'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:25.795004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 719/1425] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:48:25 2022] DELETE /identity/v3/users/d487fb82bf4344239c8cbd841e39ece3 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:25.799612 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee94390-c5e0-41c0-8578-302c9c23ba7b None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:25.799906 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee94390-c5e0-41c0-8578-302c9c23ba7b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:25.800231 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee94390-c5e0-41c0-8578-302c9c23ba7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:25.877089 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-5ee94390-c5e0-41c0-8578-302c9c23ba7b None None] MFA Rules not processed for user `cb0d97d08f9746de97f2faa0281e5696`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:25.880155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-5ee94390-c5e0-41c0-8578-302c9c23ba7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:25.939670 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2481ea6-6bd1-4323-89cc-5b35bd30ffb6 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:26.021483 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:26.051254 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 720/1426] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:26.064674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee94390-c5e0-41c0-8578-302c9c23ba7b None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:26.066058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-5ee94390-c5e0-41c0-8578-302c9c23ba7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:26.164216 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c89ea-7b34-4ef5-9611-17d3193ff06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:26.166733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c89ea-7b34-4ef5-9611-17d3193ff06a admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:26.166917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c89ea-7b34-4ef5-9611-17d3193ff06a admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:26.167299 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c89ea-7b34-4ef5-9611-17d3193ff06a admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:26.168443 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c89ea-7b34-4ef5-9611-17d3193ff06a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:26.182228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c89ea-7b34-4ef5-9611-17d3193ff06a admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:26.237438 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee94390-c5e0-41c0-8578-302c9c23ba7b None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:26.254298 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e709ef-9946-4f7e-aa52-a9304575989b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:26.254948 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e709ef-9946-4f7e-aa52-a9304575989b service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:26.255231 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e709ef-9946-4f7e-aa52-a9304575989b service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:26.255504 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e709ef-9946-4f7e-aa52-a9304575989b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:26.259864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f5e709ef-9946-4f7e-aa52-a9304575989b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:26.270273 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 707/1427] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:25 2022] POST /identity/v3/projects => generated 369 bytes in 596 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:26.271238 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c89ea-7b34-4ef5-9611-17d3193ff06a admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:26.287234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-3a4c89ea-7b34-4ef5-9611-17d3193ff06a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:26.422734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:26.480966 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e709ef-9946-4f7e-aa52-a9304575989b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:26.484930 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e709ef-9946-4f7e-aa52-a9304575989b service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:26.535903 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 721/1428] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:26.558442 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c89ea-7b34-4ef5-9611-17d3193ff06a admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:26.588712 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9dea15-03bb-4b17-9e9a-aeb1928cac4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:26.589299 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dea15-03bb-4b17-9e9a-aeb1928cac4e admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:26.589581 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dea15-03bb-4b17-9e9a-aeb1928cac4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:26.589866 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dea15-03bb-4b17-9e9a-aeb1928cac4e admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:26.590518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9dea15-03bb-4b17-9e9a-aeb1928cac4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:26.595161 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9dea15-03bb-4b17-9e9a-aeb1928cac4e admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:26.705541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e709ef-9946-4f7e-aa52-a9304575989b service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:26.708773 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 708/1429] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:26 2022] POST /identity/v3/users => generated 285 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:26.724959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e709ef-9946-4f7e-aa52-a9304575989b service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:26.757995 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495283d-eb3a-466e-9344-93cc35f8871e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:26.758789 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495283d-eb3a-466e-9344-93cc35f8871e admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:26.759223 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495283d-eb3a-466e-9344-93cc35f8871e admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:26.759535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495283d-eb3a-466e-9344-93cc35f8871e admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:26.760377 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495283d-eb3a-466e-9344-93cc35f8871e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:26.771079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495283d-eb3a-466e-9344-93cc35f8871e admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:26.817421 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 722/1430] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:26.821394 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9dea15-03bb-4b17-9e9a-aeb1928cac4e admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:26.839728 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9dea15-03bb-4b17-9e9a-aeb1928cac4e admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:26.858050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91af021-5af0-49b2-965d-36e1dda86c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:26.858857 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91af021-5af0-49b2-965d-36e1dda86c92 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:26.864298 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91af021-5af0-49b2-965d-36e1dda86c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:26.864649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91af021-5af0-49b2-965d-36e1dda86c92 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:26.865360 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91af021-5af0-49b2-965d-36e1dda86c92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:26.872831 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91af021-5af0-49b2-965d-36e1dda86c92 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:26.906595 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 709/1431] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:48:26.912885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495283d-eb3a-466e-9344-93cc35f8871e admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:26.927902 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495283d-eb3a-466e-9344-93cc35f8871e admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:26.956464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af90e35-d076-4d14-818a-f9f5a9e5f752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:26.957107 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af90e35-d076-4d14-818a-f9f5a9e5f752 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:26.957387 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af90e35-d076-4d14-818a-f9f5a9e5f752 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:26.957668 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af90e35-d076-4d14-818a-f9f5a9e5f752 admin admin] PATH_INFO: `/v3/projects/0175d72d8ca24f2b89ff29ea4d539148/users/616ff6455ed44d7f9983d75ea609b4e3/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:27.014483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af90e35-d076-4d14-818a-f9f5a9e5f752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0175d72d8ca24f2b89ff29ea4d539148, user_id=616ff6455ed44d7f9983d75ea609b4e3, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:27.017746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af90e35-d076-4d14-818a-f9f5a9e5f752 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:27.074668 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 723/1432] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:26 2022] PUT /identity/v3/projects/0175d72d8ca24f2b89ff29ea4d539148/users/616ff6455ed44d7f9983d75ea609b4e3/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:27.086226 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76cb6f-6ba9-4374-b369-31f6919525d7 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:27.086500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76cb6f-6ba9-4374-b369-31f6919525d7 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:27.086778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76cb6f-6ba9-4374-b369-31f6919525d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:27.100038 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b8d9c-4b61-4652-8d85-5c96dee4cce8 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:27.100602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b8d9c-4b61-4652-8d85-5c96dee4cce8 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:27.101016 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b8d9c-4b61-4652-8d85-5c96dee4cce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:27.176878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-860b8d9c-4b61-4652-8d85-5c96dee4cce8 None None] MFA Rules not processed for user `616ff6455ed44d7f9983d75ea609b4e3`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:27.182272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-860b8d9c-4b61-4652-8d85-5c96dee4cce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:27.193470 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-df76cb6f-6ba9-4374-b369-31f6919525d7 None None] MFA Rules not processed for user `8135391c507d4f06aeb7e74160bbd038`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:27.197466 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-df76cb6f-6ba9-4374-b369-31f6919525d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:27.399964 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 710/1433] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:27.401488 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a612e-ee1d-4431-a5c4-3251aa5e14e5 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:27.404024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a612e-ee1d-4431-a5c4-3251aa5e14e5 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:27.420737 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a612e-ee1d-4431-a5c4-3251aa5e14e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:27.429684 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 724/1434] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:27.433875 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b8d9c-4b61-4652-8d85-5c96dee4cce8 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:27.457296 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-794a612e-ee1d-4431-a5c4-3251aa5e14e5 None None] MFA Rules not processed for user `8135391c507d4f06aeb7e74160bbd038`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:27.459652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-794a612e-ee1d-4431-a5c4-3251aa5e14e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:27.468433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 711/1435] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:27.480048 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a612e-ee1d-4431-a5c4-3251aa5e14e5 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:27.484553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-794a612e-ee1d-4431-a5c4-3251aa5e14e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:27.551289 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:27.604760 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b8d9c-4b61-4652-8d85-5c96dee4cce8 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:27.611929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a4a75-8a69-4577-a66d-3d0560e6eca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:27.612919 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a4a75-8a69-4577-a66d-3d0560e6eca4 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:27.613412 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a4a75-8a69-4577-a66d-3d0560e6eca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:27.613890 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a4a75-8a69-4577-a66d-3d0560e6eca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:27.614991 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a4a75-8a69-4577-a66d-3d0560e6eca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:27.617367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a4a75-8a69-4577-a66d-3d0560e6eca4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:27.658266 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 725/1436] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:27 2022] POST /identity/v3/projects => generated 367 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:48:27.664366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a4a75-8a69-4577-a66d-3d0560e6eca4 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:27.703927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:27.729484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a612e-ee1d-4431-a5c4-3251aa5e14e5 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:27.766673 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a4a75-8a69-4577-a66d-3d0560e6eca4 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:27.777574 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79166962-95bc-4917-899e-dc9827db7a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:27.778380 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79166962-95bc-4917-899e-dc9827db7a2a admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:27.778757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79166962-95bc-4917-899e-dc9827db7a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:27.779132 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79166962-95bc-4917-899e-dc9827db7a2a admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:27.781019 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79166962-95bc-4917-899e-dc9827db7a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:27.788676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79166962-95bc-4917-899e-dc9827db7a2a admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:27.851541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 726/1437] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:27 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:27.855008 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79166962-95bc-4917-899e-dc9827db7a2a admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:27.871578 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea87ca7-09ad-45f5-829f-4ca9774e6262 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:27.873758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea87ca7-09ad-45f5-829f-4ca9774e6262 service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:27.874316 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea87ca7-09ad-45f5-829f-4ca9774e6262 service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:27.874789 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea87ca7-09ad-45f5-829f-4ca9774e6262 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:27.892601 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79166962-95bc-4917-899e-dc9827db7a2a admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:27.905527 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a312e5-e2e9-42c3-bb1f-1316db50b600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:27.906540 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a312e5-e2e9-42c3-bb1f-1316db50b600 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:27.906823 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a312e5-e2e9-42c3-bb1f-1316db50b600 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:27.907094 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a312e5-e2e9-42c3-bb1f-1316db50b600 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:27.907746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a312e5-e2e9-42c3-bb1f-1316db50b600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:27.917877 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a312e5-e2e9-42c3-bb1f-1316db50b600 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:27.948734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 727/1438] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:27.949931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a312e5-e2e9-42c3-bb1f-1316db50b600 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:27.965796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a312e5-e2e9-42c3-bb1f-1316db50b600 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:27.981317 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad977c73-4f1b-4c5c-b4cb-5b2e8533876d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:27.984333 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad977c73-4f1b-4c5c-b4cb-5b2e8533876d admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:27.984333 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad977c73-4f1b-4c5c-b4cb-5b2e8533876d admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:27.984533 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad977c73-4f1b-4c5c-b4cb-5b2e8533876d admin admin] PATH_INFO: `/v3/projects/7db9fa19df73499d843c7b35034aa7a7/users/158114d85ff64e518778b409bd7e5691/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:28.040467 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea87ca7-09ad-45f5-829f-4ca9774e6262 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:28.044599 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea87ca7-09ad-45f5-829f-4ca9774e6262 service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:28.052552 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad977c73-4f1b-4c5c-b4cb-5b2e8533876d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7db9fa19df73499d843c7b35034aa7a7, user_id=158114d85ff64e518778b409bd7e5691, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:28.057799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad977c73-4f1b-4c5c-b4cb-5b2e8533876d admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:28.075690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 712/1439] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:28.079653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f09b0b-22e2-47f4-bbe6-243a4bd8b8e3 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:28.079952 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f09b0b-22e2-47f4-bbe6-243a4bd8b8e3 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:28.083547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f09b0b-22e2-47f4-bbe6-243a4bd8b8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:28.104213 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 728/1440] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:27 2022] PUT /identity/v3/projects/7db9fa19df73499d843c7b35034aa7a7/users/158114d85ff64e518778b409bd7e5691/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:28.105682 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad977c73-4f1b-4c5c-b4cb-5b2e8533876d admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:28.122522 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ad977c73-4f1b-4c5c-b4cb-5b2e8533876d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:28.159873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-e1f09b0b-22e2-47f4-bbe6-243a4bd8b8e3 None None] MFA Rules not processed for user `350f5e4238f448949b2c372aabbfc835`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:28.162552 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e1f09b0b-22e2-47f4-bbe6-243a4bd8b8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:28.281916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:28.284783 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad977c73-4f1b-4c5c-b4cb-5b2e8533876d admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:28.297643 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc348fe0-6f19-4345-a26c-8aeb93ee9a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:28.298382 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc348fe0-6f19-4345-a26c-8aeb93ee9a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:28.299643 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc348fe0-6f19-4345-a26c-8aeb93ee9a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:28.299997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc348fe0-6f19-4345-a26c-8aeb93ee9a3d admin admin] PATH_INFO: `/v3/projects/3ee356ebd4a048bb99322a3219c25020` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:28.302277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc348fe0-6f19-4345-a26c-8aeb93ee9a3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ee356ebd4a048bb99322a3219c25020)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:28.307683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc348fe0-6f19-4345-a26c-8aeb93ee9a3d admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:28.345187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 713/1441] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:28.348318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f09b0b-22e2-47f4-bbe6-243a4bd8b8e3 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:28.372489 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f09b0b-22e2-47f4-bbe6-243a4bd8b8e3 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:28.385021 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f951378-4840-4791-9042-802f5fef45d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:28.387100 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f951378-4840-4791-9042-802f5fef45d4 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:28.387542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f951378-4840-4791-9042-802f5fef45d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:28.390241 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f951378-4840-4791-9042-802f5fef45d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:28.391092 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f951378-4840-4791-9042-802f5fef45d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:28.394560 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f951378-4840-4791-9042-802f5fef45d4 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:28.418611 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 714/1442] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:28.452871 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f95a4-f45d-4867-a14a-695ac969492f None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:28.453480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f95a4-f45d-4867-a14a-695ac969492f None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:28.459208 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f95a4-f45d-4867-a14a-695ac969492f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:28.508641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-dc348fe0-6f19-4345-a26c-8aeb93ee9a3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ee356ebd4a048bb99322a3219c25020', 'request_id': 'req-dc348fe0-6f19-4345-a26c-8aeb93ee9a3d'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:28.524193 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-dc348fe0-6f19-4345-a26c-8aeb93ee9a3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ee356ebd4a048bb99322a3219c25020', 'request_id': 'req-dc348fe0-6f19-4345-a26c-8aeb93ee9a3d'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:28.533732 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-dc348fe0-6f19-4345-a26c-8aeb93ee9a3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ee356ebd4a048bb99322a3219c25020', 'request_id': 'req-dc348fe0-6f19-4345-a26c-8aeb93ee9a3d'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:28.556121 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-dc348fe0-6f19-4345-a26c-8aeb93ee9a3d admin admin] The token cache is being invalidate because project 3ee356ebd4a048bb99322a3219c25020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:48:28.558544 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-dc348fe0-6f19-4345-a26c-8aeb93ee9a3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:28.559396 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-dc348fe0-6f19-4345-a26c-8aeb93ee9a3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:28.565989 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 729/1443] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:48:28 2022] DELETE /identity/v3/projects/3ee356ebd4a048bb99322a3219c25020 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:28.568935 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0ddd0-e1ee-40d0-9d53-ab811e9edee4 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:28.569402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0ddd0-e1ee-40d0-9d53-ab811e9edee4 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:28.570303 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0ddd0-e1ee-40d0-9d53-ab811e9edee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:28.571371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-729f95a4-f45d-4867-a14a-695ac969492f None None] MFA Rules not processed for user `5ac3fdb86ac24fa493706f84e1aa40a8`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:28.572567 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-729f95a4-f45d-4867-a14a-695ac969492f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:28.594071 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-c7d0ddd0-e1ee-40d0-9d53-ab811e9edee4 None None] MFA Rules not processed for user `350f5e4238f448949b2c372aabbfc835`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:28.595918 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c7d0ddd0-e1ee-40d0-9d53-ab811e9edee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:28.601216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:28.710522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 715/1444] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:28.711381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f95a4-f45d-4867-a14a-695ac969492f None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:28.713189 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-729f95a4-f45d-4867-a14a-695ac969492f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:28.743509 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 730/1445] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:28.745349 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b82306-0f87-4022-9689-96830298e4ca None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:28.745633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b82306-0f87-4022-9689-96830298e4ca None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:28.745909 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b82306-0f87-4022-9689-96830298e4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:28.802198 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-32b82306-0f87-4022-9689-96830298e4ca None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:28.804017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-32b82306-0f87-4022-9689-96830298e4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:28.853733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f95a4-f45d-4867-a14a-695ac969492f None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:28.861028 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61501a3-e661-4f6b-bd97-1d29df45dab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:28.861205 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61501a3-e661-4f6b-bd97-1d29df45dab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:28.861436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61501a3-e661-4f6b-bd97-1d29df45dab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:28.861668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61501a3-e661-4f6b-bd97-1d29df45dab5 admin admin] PATH_INFO: `/v3/projects/7db9fa19df73499d843c7b35034aa7a7/users/158114d85ff64e518778b409bd7e5691/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:28.863917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61501a3-e661-4f6b-bd97-1d29df45dab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7db9fa19df73499d843c7b35034aa7a7, user_id=158114d85ff64e518778b409bd7e5691, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:28.867494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61501a3-e661-4f6b-bd97-1d29df45dab5 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:28.896623 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 716/1446] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:28 2022] PUT /identity/v3/projects/7db9fa19df73499d843c7b35034aa7a7/users/158114d85ff64e518778b409bd7e5691/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:28.896623 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61501a3-e661-4f6b-bd97-1d29df45dab5 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:28.901093 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d61501a3-e661-4f6b-bd97-1d29df45dab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:28.904820 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 731/1447] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:28.908249 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81604e7-d9ff-4de4-9ea0-ffd77e9ecf10 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:28.909080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81604e7-d9ff-4de4-9ea0-ffd77e9ecf10 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:28.909080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81604e7-d9ff-4de4-9ea0-ffd77e9ecf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:28.950843 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-e81604e7-d9ff-4de4-9ea0-ffd77e9ecf10 None None] MFA Rules not processed for user `158114d85ff64e518778b409bd7e5691`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:28.967190 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e81604e7-d9ff-4de4-9ea0-ffd77e9ecf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:28.993767 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:29.037051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61501a3-e661-4f6b-bd97-1d29df45dab5 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:29.046880 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa25f7b-55e3-4ff3-8b8f-1a72bcd9058d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:29.047815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa25f7b-55e3-4ff3-8b8f-1a72bcd9058d service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:29.048319 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa25f7b-55e3-4ff3-8b8f-1a72bcd9058d service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:29.048766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa25f7b-55e3-4ff3-8b8f-1a72bcd9058d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:29.073687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 732/1448] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:29.082980 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a5d74-a4e7-47c5-b0cf-7ffb57d9e409 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:29.083624 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a5d74-a4e7-47c5-b0cf-7ffb57d9e409 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:29.084053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a5d74-a4e7-47c5-b0cf-7ffb57d9e409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:29.107172 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-427a5d74-a4e7-47c5-b0cf-7ffb57d9e409 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:29.109089 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-427a5d74-a4e7-47c5-b0cf-7ffb57d9e409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:29.163053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:29.198231 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa25f7b-55e3-4ff3-8b8f-1a72bcd9058d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:29.200456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa25f7b-55e3-4ff3-8b8f-1a72bcd9058d service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:29.203822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 733/1449] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:29.205663 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a5d74-a4e7-47c5-b0cf-7ffb57d9e409 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:29.207517 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-427a5d74-a4e7-47c5-b0cf-7ffb57d9e409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:29.234783 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 717/1450] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:29.235784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa25f7b-55e3-4ff3-8b8f-1a72bcd9058d service nova] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:29.268648 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa25f7b-55e3-4ff3-8b8f-1a72bcd9058d service nova] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:29.288835 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a3660-9511-4390-a80c-cb50954a41c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:29.290126 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a3660-9511-4390-a80c-cb50954a41c1 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:29.291126 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a3660-9511-4390-a80c-cb50954a41c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:29.291547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a3660-9511-4390-a80c-cb50954a41c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:29.294996 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8a3660-9511-4390-a80c-cb50954a41c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:29.295494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8a3660-9511-4390-a80c-cb50954a41c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:29.299476 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8a3660-9511-4390-a80c-cb50954a41c1 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:29.359719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 718/1451] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:29.363291 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a3660-9511-4390-a80c-cb50954a41c1 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:29.387415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a3660-9511-4390-a80c-cb50954a41c1 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:29.400363 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88aba2-b780-4629-af11-336b0049f297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:29.402490 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88aba2-b780-4629-af11-336b0049f297 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:29.402826 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88aba2-b780-4629-af11-336b0049f297 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:29.403168 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88aba2-b780-4629-af11-336b0049f297 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:29.403269 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a5d74-a4e7-47c5-b0cf-7ffb57d9e409 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:29.403739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e88aba2-b780-4629-af11-336b0049f297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:29.407141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e88aba2-b780-4629-af11-336b0049f297 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:29.415239 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143d63c-cd8a-462d-a7f0-249b05f8e7ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:29.423089 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143d63c-cd8a-462d-a7f0-249b05f8e7ef service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:29.424314 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143d63c-cd8a-462d-a7f0-249b05f8e7ef service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:29.424603 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143d63c-cd8a-462d-a7f0-249b05f8e7ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:29.459312 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143d63c-cd8a-462d-a7f0-249b05f8e7ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:29.460942 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143d63c-cd8a-462d-a7f0-249b05f8e7ef service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:29.478998 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 734/1452] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:29.479845 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143d63c-cd8a-462d-a7f0-249b05f8e7ef service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:29.482052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c143d63c-cd8a-462d-a7f0-249b05f8e7ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:29.494613 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 719/1453] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:29 2022] POST /identity/v3/projects => generated 369 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:29.501232 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88aba2-b780-4629-af11-336b0049f297 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:29.506853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143d63c-cd8a-462d-a7f0-249b05f8e7ef service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:29.704969 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88aba2-b780-4629-af11-336b0049f297 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:29.705983 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4669e-8ae5-4967-aeb7-781785571133 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:29.708340 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4669e-8ae5-4967-aeb7-781785571133 service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:29.709694 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4669e-8ae5-4967-aeb7-781785571133 service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:29.709968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4669e-8ae5-4967-aeb7-781785571133 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:29.718259 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-0dd4669e-8ae5-4967-aeb7-781785571133 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:29.729882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f071e-a971-443d-a4ed-4b6735c0b0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:29.730589 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f071e-a971-443d-a4ed-4b6735c0b0a8 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:29.731211 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f071e-a971-443d-a4ed-4b6735c0b0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:29.731211 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f071e-a971-443d-a4ed-4b6735c0b0a8 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:29.742633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f071e-a971-443d-a4ed-4b6735c0b0a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:29.748586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f071e-a971-443d-a4ed-4b6735c0b0a8 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:29.813588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 720/1454] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:29 2022] POST /identity/v3/users => generated 285 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:48:29.816999 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6107a41-acdf-42d1-be6a-ee92c2c7e221 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:29.817301 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6107a41-acdf-42d1-be6a-ee92c2c7e221 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:29.817581 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6107a41-acdf-42d1-be6a-ee92c2c7e221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:29.939679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-c6107a41-acdf-42d1-be6a-ee92c2c7e221 None None] MFA Rules not processed for user `8a91d3b9c4834d729211ec367e235889`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:29.941021 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c6107a41-acdf-42d1-be6a-ee92c2c7e221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:29.976861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:30.010999 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4669e-8ae5-4967-aeb7-781785571133 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:30.014862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4669e-8ae5-4967-aeb7-781785571133 service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:30.063751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 735/1455] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:29 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:30.065039 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4669e-8ae5-4967-aeb7-781785571133 service nova] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:30.094436 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4669e-8ae5-4967-aeb7-781785571133 service nova] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:30.120305 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e46b6-a7f8-401d-9eba-03e4536b5b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:30.120305 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e46b6-a7f8-401d-9eba-03e4536b5b36 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:30.120305 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e46b6-a7f8-401d-9eba-03e4536b5b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:30.120305 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e46b6-a7f8-401d-9eba-03e4536b5b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:30.122113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e46b6-a7f8-401d-9eba-03e4536b5b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:30.129903 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e46b6-a7f8-401d-9eba-03e4536b5b36 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:30.130592 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 721/1456] 10.222.0.36 () {58 vars in 1009 bytes} [Sun Oct 30 00:48:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:30.180371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 736/1457] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:30.184694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6107a41-acdf-42d1-be6a-ee92c2c7e221 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:30.226160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6107a41-acdf-42d1-be6a-ee92c2c7e221 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:30.241164 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222ba9f-1e01-434b-a45a-ecaf2e726afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:30.242390 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222ba9f-1e01-434b-a45a-ecaf2e726afa admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:30.248901 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222ba9f-1e01-434b-a45a-ecaf2e726afa admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:30.249048 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222ba9f-1e01-434b-a45a-ecaf2e726afa admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:30.249741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222ba9f-1e01-434b-a45a-ecaf2e726afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:30.257630 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222ba9f-1e01-434b-a45a-ecaf2e726afa admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:30.284626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 722/1458] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:48:30.297804 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e46b6-a7f8-401d-9eba-03e4536b5b36 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:30.322831 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e46b6-a7f8-401d-9eba-03e4536b5b36 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:30.346549 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157f94d-8c86-4a4c-a5ca-6102e1fbdbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:30.347557 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157f94d-8c86-4a4c-a5ca-6102e1fbdbd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:30.348931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157f94d-8c86-4a4c-a5ca-6102e1fbdbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:30.356297 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157f94d-8c86-4a4c-a5ca-6102e1fbdbd7 admin admin] PATH_INFO: `/v3/projects/5fbee574177440a7a2245f756d84f845/users/0d0b3e1f744b475ebd1f490df5af4679/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:30.369777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:30.425398 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157f94d-8c86-4a4c-a5ca-6102e1fbdbd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fbee574177440a7a2245f756d84f845, user_id=0d0b3e1f744b475ebd1f490df5af4679, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:30.428457 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157f94d-8c86-4a4c-a5ca-6102e1fbdbd7 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:30.474740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222ba9f-1e01-434b-a45a-ecaf2e726afa admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:30.477443 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4222ba9f-1e01-434b-a45a-ecaf2e726afa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:30.513188 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 737/1459] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:30 2022] PUT /identity/v3/projects/5fbee574177440a7a2245f756d84f845/users/0d0b3e1f744b475ebd1f490df5af4679/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:30.515318 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fe166-8caa-4ef5-adf7-7648c3bc2162 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:30.515606 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fe166-8caa-4ef5-adf7-7648c3bc2162 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:30.515882 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fe166-8caa-4ef5-adf7-7648c3bc2162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:30.560055 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-369fe166-8caa-4ef5-adf7-7648c3bc2162 None None] MFA Rules not processed for user `0d0b3e1f744b475ebd1f490df5af4679`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:30.566755 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-369fe166-8caa-4ef5-adf7-7648c3bc2162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:30.687689 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 738/1460] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:30.692798 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369fe166-8caa-4ef5-adf7-7648c3bc2162 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:30.816345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369fe166-8caa-4ef5-adf7-7648c3bc2162 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:30.828644 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609fe06-d737-4484-bc98-44e62feb87df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:30.829443 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609fe06-d737-4484-bc98-44e62feb87df admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:30.829729 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609fe06-d737-4484-bc98-44e62feb87df admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:30.830151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609fe06-d737-4484-bc98-44e62feb87df admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:30.830823 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609fe06-d737-4484-bc98-44e62feb87df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:30.833626 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609fe06-d737-4484-bc98-44e62feb87df admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:30.925142 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 739/1461] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:30 2022] POST /identity/v3/projects => generated 369 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:48:30.926970 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609fe06-d737-4484-bc98-44e62feb87df admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:30.936519 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-9609fe06-d737-4484-bc98-44e62feb87df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:30.981295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222ba9f-1e01-434b-a45a-ecaf2e726afa admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:31.099634 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609fe06-d737-4484-bc98-44e62feb87df admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:31.103567 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266d2e6-2c4b-400b-85ad-38c90d5be2fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:31.104748 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266d2e6-2c4b-400b-85ad-38c90d5be2fd service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:31.105029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266d2e6-2c4b-400b-85ad-38c90d5be2fd service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:31.105305 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266d2e6-2c4b-400b-85ad-38c90d5be2fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:31.112945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4266d2e6-2c4b-400b-85ad-38c90d5be2fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:31.144122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:31.155238 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556f5bc-b2e0-482b-87b6-7d21ff25be6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f4c779b8422432fb06cdf996841b9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:31.160030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556f5bc-b2e0-482b-87b6-7d21ff25be6c service swift] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:31.179911 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556f5bc-b2e0-482b-87b6-7d21ff25be6c service swift] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:31.181531 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556f5bc-b2e0-482b-87b6-7d21ff25be6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:31.237781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266d2e6-2c4b-400b-85ad-38c90d5be2fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:31.239615 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266d2e6-2c4b-400b-85ad-38c90d5be2fd service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:31.257777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 740/1462] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:31.258922 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266d2e6-2c4b-400b-85ad-38c90d5be2fd service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:31.313720 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:31.373550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556f5bc-b2e0-482b-87b6-7d21ff25be6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:31.378331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556f5bc-b2e0-482b-87b6-7d21ff25be6c service swift] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:31.423718 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 723/1463] 10.222.0.36 () {60 vars in 1384 bytes} [Sun Oct 30 00:48:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 949 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 30 00:48:31.495964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266d2e6-2c4b-400b-85ad-38c90d5be2fd service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:31.521053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5e615-39f9-4317-8bd1-3496fb5c247d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:31.521595 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5e615-39f9-4317-8bd1-3496fb5c247d admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:31.521920 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5e615-39f9-4317-8bd1-3496fb5c247d admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:31.529708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5e615-39f9-4317-8bd1-3496fb5c247d admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:31.530504 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5e615-39f9-4317-8bd1-3496fb5c247d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:31.546243 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5e615-39f9-4317-8bd1-3496fb5c247d admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:31.626171 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 741/1464] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:31 2022] POST /identity/v3/users => generated 291 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:31.628944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556f5bc-b2e0-482b-87b6-7d21ff25be6c service swift] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:31.647927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556f5bc-b2e0-482b-87b6-7d21ff25be6c service swift] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:31.683793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4a0ac-bf8d-4584-bf0e-742ef206f3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:31.684816 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4a0ac-bf8d-4584-bf0e-742ef206f3e1 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:31.685793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4a0ac-bf8d-4584-bf0e-742ef206f3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:31.686831 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4a0ac-bf8d-4584-bf0e-742ef206f3e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:31.689615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c4a0ac-bf8d-4584-bf0e-742ef206f3e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:31.698947 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c4a0ac-bf8d-4584-bf0e-742ef206f3e1 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:31.763231 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 724/1465] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:31.767854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5e615-39f9-4317-8bd1-3496fb5c247d admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:31.791547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5e615-39f9-4317-8bd1-3496fb5c247d admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:31.826446 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7beab-361c-43fa-8c5f-ce2be059909a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:31.827626 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7beab-361c-43fa-8c5f-ce2be059909a admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:31.830509 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7beab-361c-43fa-8c5f-ce2be059909a admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:31.835262 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7beab-361c-43fa-8c5f-ce2be059909a admin admin] PATH_INFO: `/v3/projects/cb47324846064bcdba1d823141034334/users/eafa63c27141480bb35a424ce44ae880/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:31.901314 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f7beab-361c-43fa-8c5f-ce2be059909a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb47324846064bcdba1d823141034334, user_id=eafa63c27141480bb35a424ce44ae880, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:31.910403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f7beab-361c-43fa-8c5f-ce2be059909a admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:31.975045 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 742/1466] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:31 2022] PUT /identity/v3/projects/cb47324846064bcdba1d823141034334/users/eafa63c27141480bb35a424ce44ae880/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:48:31.979160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4a0ac-bf8d-4584-bf0e-742ef206f3e1 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:31.999785 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4a0ac-bf8d-4584-bf0e-742ef206f3e1 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:32.258216 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb30bd3-6b22-4d0a-a6ae-e9ad1fb49a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:32.258859 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb30bd3-6b22-4d0a-a6ae-e9ad1fb49a70 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:32.259237 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb30bd3-6b22-4d0a-a6ae-e9ad1fb49a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:32.259523 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb30bd3-6b22-4d0a-a6ae-e9ad1fb49a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:32.260209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb30bd3-6b22-4d0a-a6ae-e9ad1fb49a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:32.262087 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb30bd3-6b22-4d0a-a6ae-e9ad1fb49a70 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:32.308000 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7beab-361c-43fa-8c5f-ce2be059909a admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:32.308980 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 725/1467] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:32.331013 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7beab-361c-43fa-8c5f-ce2be059909a admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:32.357035 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9a833-34c5-4d88-8172-ab9c25b7f695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:32.359022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9a833-34c5-4d88-8172-ab9c25b7f695 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:32.359506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9a833-34c5-4d88-8172-ab9c25b7f695 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:32.360047 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9a833-34c5-4d88-8172-ab9c25b7f695 admin admin] PATH_INFO: `/v3/projects/cb47324846064bcdba1d823141034334/users/eafa63c27141480bb35a424ce44ae880/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:32.367343 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9a833-34c5-4d88-8172-ab9c25b7f695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb47324846064bcdba1d823141034334, user_id=eafa63c27141480bb35a424ce44ae880, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:32.381898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9a833-34c5-4d88-8172-ab9c25b7f695 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:32.434555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 743/1468] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:32 2022] PUT /identity/v3/projects/cb47324846064bcdba1d823141034334/users/eafa63c27141480bb35a424ce44ae880/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:48:32.446544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007b03c-348e-4223-9d6b-2c901eb89c76 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:32.446762 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007b03c-348e-4223-9d6b-2c901eb89c76 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:32.447237 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007b03c-348e-4223-9d6b-2c901eb89c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:32.556647 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-7007b03c-348e-4223-9d6b-2c901eb89c76 None None] MFA Rules not processed for user `eafa63c27141480bb35a424ce44ae880`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:32.558093 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-7007b03c-348e-4223-9d6b-2c901eb89c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:32.906044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 726/1469] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:33.060348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9a833-34c5-4d88-8172-ab9c25b7f695 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:33.356133 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9a833-34c5-4d88-8172-ab9c25b7f695 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:33.378499 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7730e05-6667-41fe-a492-c4167e19230c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:33.381769 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7730e05-6667-41fe-a492-c4167e19230c service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:33.382887 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7730e05-6667-41fe-a492-c4167e19230c service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:33.384151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7730e05-6667-41fe-a492-c4167e19230c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:33.415697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7730e05-6667-41fe-a492-c4167e19230c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:33.420093 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7730e05-6667-41fe-a492-c4167e19230c service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:33.463148 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 744/1470] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:33 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:33.758150 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601248f-9a19-4537-8418-88ae7407cf52 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:33.758645 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601248f-9a19-4537-8418-88ae7407cf52 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:33.758951 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601248f-9a19-4537-8418-88ae7407cf52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:33.848917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-0601248f-9a19-4537-8418-88ae7407cf52 None None] MFA Rules not processed for user `350f5e4238f448949b2c372aabbfc835`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:33.864167 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-0601248f-9a19-4537-8418-88ae7407cf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:33.883170 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7730e05-6667-41fe-a492-c4167e19230c service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:33.908150 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7730e05-6667-41fe-a492-c4167e19230c service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:33.928100 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd725af-f81b-430f-a32f-e1a4e5144791 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:33.928902 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd725af-f81b-430f-a32f-e1a4e5144791 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:33.929337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd725af-f81b-430f-a32f-e1a4e5144791 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:33.930171 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd725af-f81b-430f-a32f-e1a4e5144791 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:33.953374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:34.103381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:34.124193 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd725af-f81b-430f-a32f-e1a4e5144791 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:34.126238 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd725af-f81b-430f-a32f-e1a4e5144791 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:34.141185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 745/1471] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:34.164280 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd725af-f81b-430f-a32f-e1a4e5144791 service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:34.180766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 727/1472] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:34.199014 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd725af-f81b-430f-a32f-e1a4e5144791 service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:34.220171 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fd8d3-a2b7-4a43-a353-b388c2731dbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:34.224107 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fd8d3-a2b7-4a43-a353-b388c2731dbd service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:34.224780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fd8d3-a2b7-4a43-a353-b388c2731dbd service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:34.225630 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fd8d3-a2b7-4a43-a353-b388c2731dbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:34.232758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-224fd8d3-a2b7-4a43-a353-b388c2731dbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:34.505333 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fd8d3-a2b7-4a43-a353-b388c2731dbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:34.507742 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fd8d3-a2b7-4a43-a353-b388c2731dbd service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:34.521997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 746/1473] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:35.019271 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df7650-ee11-44c2-b139-3e30cbd9006d None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:35.020285 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df7650-ee11-44c2-b139-3e30cbd9006d None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:35.020716 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df7650-ee11-44c2-b139-3e30cbd9006d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:35.102559 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-98df7650-ee11-44c2-b139-3e30cbd9006d None None] MFA Rules not processed for user `fb39b6cc97a1456bb1312f49118c6e10`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:35.119737 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-98df7650-ee11-44c2-b139-3e30cbd9006d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:35.306262 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 728/1474] 10.222.0.20 () {58 vars in 1015 bytes} [Sun Oct 30 00:48:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:35.338271 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fd8d3-a2b7-4a43-a353-b388c2731dbd service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:35.341117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-224fd8d3-a2b7-4a43-a353-b388c2731dbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:35.370709 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fd8d3-a2b7-4a43-a353-b388c2731dbd service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:35.417478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fe269-30c5-4c8a-8b4a-8511b5b67923 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:35.418118 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fe269-30c5-4c8a-8b4a-8511b5b67923 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:35.418389 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fe269-30c5-4c8a-8b4a-8511b5b67923 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:35.418670 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fe269-30c5-4c8a-8b4a-8511b5b67923 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:35.456657 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5fe269-30c5-4c8a-8b4a-8511b5b67923 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:35.461057 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5fe269-30c5-4c8a-8b4a-8511b5b67923 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:35.489962 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 747/1475] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:37.116071 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455793cd-45fd-4f22-9720-9bc38969842b None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:37.116609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455793cd-45fd-4f22-9720-9bc38969842b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:37.116609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455793cd-45fd-4f22-9720-9bc38969842b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:37.227733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-455793cd-45fd-4f22-9720-9bc38969842b None None] MFA Rules not processed for user `5ac3fdb86ac24fa493706f84e1aa40a8`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:37.231171 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-455793cd-45fd-4f22-9720-9bc38969842b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:37.537972 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 729/1476] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:39.194574 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7a0bc-71c9-4437-96b7-0d0068d16ce9 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:39.195000 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7a0bc-71c9-4437-96b7-0d0068d16ce9 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:39.195316 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7a0bc-71c9-4437-96b7-0d0068d16ce9 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:39.198668 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 748/1477] 10.222.0.36 () {54 vars in 912 bytes} [Sun Oct 30 00:48:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:39.238915 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97256e26-01d6-4e60-962c-9a6923ac0547 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:39.240111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97256e26-01d6-4e60-962c-9a6923ac0547 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:39.240589 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97256e26-01d6-4e60-962c-9a6923ac0547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:39.311899 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7a0bc-71c9-4437-96b7-0d0068d16ce9 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:39.325575 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-97256e26-01d6-4e60-962c-9a6923ac0547 None None] MFA Rules not processed for user `fb39b6cc97a1456bb1312f49118c6e10`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:39.330802 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-97256e26-01d6-4e60-962c-9a6923ac0547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:39.376865 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 730/1478] 10.222.0.36 () {58 vars in 1009 bytes} [Sun Oct 30 00:48:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:39.492482 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97256e26-01d6-4e60-962c-9a6923ac0547 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:39.494494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-97256e26-01d6-4e60-962c-9a6923ac0547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:39.536507 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97256e26-01d6-4e60-962c-9a6923ac0547 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:39.569740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7a0bc-71c9-4437-96b7-0d0068d16ce9 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:39.571744 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a68a95-3898-44be-a407-c4496af6bf12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:39.572693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a68a95-3898-44be-a407-c4496af6bf12 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:39.572979 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a68a95-3898-44be-a407-c4496af6bf12 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:39.573426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a68a95-3898-44be-a407-c4496af6bf12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:39.578503 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522b985-46f3-4664-9bea-4852a8737dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:39.579161 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522b985-46f3-4664-9bea-4852a8737dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:39.579505 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522b985-46f3-4664-9bea-4852a8737dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:39.579789 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522b985-46f3-4664-9bea-4852a8737dc7 admin admin] PATH_INFO: `/v3/users/616ff6455ed44d7f9983d75ea609b4e3` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:39.581144 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522b985-46f3-4664-9bea-4852a8737dc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=616ff6455ed44d7f9983d75ea609b4e3)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:39.586592 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522b985-46f3-4664-9bea-4852a8737dc7 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:39.601616 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a68a95-3898-44be-a407-c4496af6bf12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:39.605650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a68a95-3898-44be-a407-c4496af6bf12 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:39.662225 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 731/1479] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:39.725726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:39.805192 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f522b985-46f3-4664-9bea-4852a8737dc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '616ff6455ed44d7f9983d75ea609b4e3'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:39.807694 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f522b985-46f3-4664-9bea-4852a8737dc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '616ff6455ed44d7f9983d75ea609b4e3'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:39.912265 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f522b985-46f3-4664-9bea-4852a8737dc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '616ff6455ed44d7f9983d75ea609b4e3'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:39.941946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f522b985-46f3-4664-9bea-4852a8737dc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '616ff6455ed44d7f9983d75ea609b4e3'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:39.954279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f522b985-46f3-4664-9bea-4852a8737dc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '616ff6455ed44d7f9983d75ea609b4e3'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:39.979561 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 749/1480] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:48:39 2022] DELETE /identity/v3/users/616ff6455ed44d7f9983d75ea609b4e3 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:48:39.980605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450de2e-c159-433b-99fc-73d235f3a151 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:39.981437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450de2e-c159-433b-99fc-73d235f3a151 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:39.982246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450de2e-c159-433b-99fc-73d235f3a151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:40.057175 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-b450de2e-c159-433b-99fc-73d235f3a151 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:40.059074 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b450de2e-c159-433b-99fc-73d235f3a151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:40.220112 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 732/1481] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:40.290885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522b985-46f3-4664-9bea-4852a8737dc7 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:40.292633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f522b985-46f3-4664-9bea-4852a8737dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:40.500568 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522b985-46f3-4664-9bea-4852a8737dc7 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:40.509725 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f45d8-76ad-4517-a5b0-01d1d3015b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:40.510783 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f45d8-76ad-4517-a5b0-01d1d3015b1c service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:40.511505 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f45d8-76ad-4517-a5b0-01d1d3015b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:40.512063 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f45d8-76ad-4517-a5b0-01d1d3015b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:40.528515 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f45d8-76ad-4517-a5b0-01d1d3015b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:40.530687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f45d8-76ad-4517-a5b0-01d1d3015b1c service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:40.554915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 750/1482] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:41.050469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450de2e-c159-433b-99fc-73d235f3a151 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:41.052210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b450de2e-c159-433b-99fc-73d235f3a151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:41.088379 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450de2e-c159-433b-99fc-73d235f3a151 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:41.113990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b309db5-f949-40f8-8bc9-b80227413437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:41.114658 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b309db5-f949-40f8-8bc9-b80227413437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:41.116929 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b309db5-f949-40f8-8bc9-b80227413437 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:41.116929 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b309db5-f949-40f8-8bc9-b80227413437 admin admin] PATH_INFO: `/v3/users/6b20903295fe4c638a2e7656a4ba74e1` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:41.117533 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b309db5-f949-40f8-8bc9-b80227413437 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b20903295fe4c638a2e7656a4ba74e1)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:41.120538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b309db5-f949-40f8-8bc9-b80227413437 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:41.187297 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:41.273407 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6b309db5-f949-40f8-8bc9-b80227413437 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b20903295fe4c638a2e7656a4ba74e1'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:41.275847 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6b309db5-f949-40f8-8bc9-b80227413437 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b20903295fe4c638a2e7656a4ba74e1'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:41.328641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6b309db5-f949-40f8-8bc9-b80227413437 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b20903295fe4c638a2e7656a4ba74e1'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:41.347793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6b309db5-f949-40f8-8bc9-b80227413437 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b20903295fe4c638a2e7656a4ba74e1'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:41.349998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6b309db5-f949-40f8-8bc9-b80227413437 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b20903295fe4c638a2e7656a4ba74e1'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:41.390389 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 733/1483] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:48:41 2022] DELETE /identity/v3/users/6b20903295fe4c638a2e7656a4ba74e1 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:41.392077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819acbcb-7e7a-4bd0-93aa-6d80eda4f1b2 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:41.392716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819acbcb-7e7a-4bd0-93aa-6d80eda4f1b2 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:41.393149 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819acbcb-7e7a-4bd0-93aa-6d80eda4f1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:41.494808 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-819acbcb-7e7a-4bd0-93aa-6d80eda4f1b2 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:41.496491 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-819acbcb-7e7a-4bd0-93aa-6d80eda4f1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:41.522941 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b309db5-f949-40f8-8bc9-b80227413437 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:41.525070 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-6b309db5-f949-40f8-8bc9-b80227413437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:41.677219 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 751/1484] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:48:41.708282 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819acbcb-7e7a-4bd0-93aa-6d80eda4f1b2 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:41.709879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-819acbcb-7e7a-4bd0-93aa-6d80eda4f1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:41.798656 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b309db5-f949-40f8-8bc9-b80227413437 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:41.812735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a9be5-1702-47ea-a170-c77206082d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:41.813170 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a9be5-1702-47ea-a170-c77206082d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:41.813458 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a9be5-1702-47ea-a170-c77206082d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:41.813736 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a9be5-1702-47ea-a170-c77206082d53 admin admin] PATH_INFO: `/v3/users/0d0b3e1f744b475ebd1f490df5af4679` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:41.818713 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a9be5-1702-47ea-a170-c77206082d53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d0b3e1f744b475ebd1f490df5af4679)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:41.826096 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a9be5-1702-47ea-a170-c77206082d53 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:41.942113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819acbcb-7e7a-4bd0-93aa-6d80eda4f1b2 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:41.949554 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c4056-2572-4b66-b312-b4e8f2dae4bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:41.950324 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c4056-2572-4b66-b312-b4e8f2dae4bd service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:41.951456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c4056-2572-4b66-b312-b4e8f2dae4bd service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:41.952957 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c4056-2572-4b66-b312-b4e8f2dae4bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:41.961296 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:41.969935 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c4056-2572-4b66-b312-b4e8f2dae4bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:41.972668 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c4056-2572-4b66-b312-b4e8f2dae4bd service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:41.999661 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 752/1485] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:48:42.087421 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e85a9be5-1702-47ea-a170-c77206082d53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d0b3e1f744b475ebd1f490df5af4679'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:42.088451 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e85a9be5-1702-47ea-a170-c77206082d53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d0b3e1f744b475ebd1f490df5af4679'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:42.150362 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e85a9be5-1702-47ea-a170-c77206082d53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d0b3e1f744b475ebd1f490df5af4679'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:42.173232 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e85a9be5-1702-47ea-a170-c77206082d53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d0b3e1f744b475ebd1f490df5af4679'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:42.175335 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e85a9be5-1702-47ea-a170-c77206082d53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d0b3e1f744b475ebd1f490df5af4679'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:42.237218 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 734/1486] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:48:41 2022] DELETE /identity/v3/users/0d0b3e1f744b475ebd1f490df5af4679 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:48:42.242952 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1723799-8dec-4cf4-a324-7db36b58299e None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:42.243314 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1723799-8dec-4cf4-a324-7db36b58299e None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:42.244134 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1723799-8dec-4cf4-a324-7db36b58299e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:42.282945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-e1723799-8dec-4cf4-a324-7db36b58299e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:42.287004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e1723799-8dec-4cf4-a324-7db36b58299e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:42.483512 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 753/1487] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:42.654035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a9be5-1702-47ea-a170-c77206082d53 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:42.655631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e85a9be5-1702-47ea-a170-c77206082d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:42.709963 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:42.828913 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1723799-8dec-4cf4-a324-7db36b58299e None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:42.832528 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e1723799-8dec-4cf4-a324-7db36b58299e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:42.839058 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a9be5-1702-47ea-a170-c77206082d53 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:42.850140 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863ae34-7033-4c6c-8b8e-28d8f28a92c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:42.851092 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863ae34-7033-4c6c-8b8e-28d8f28a92c0 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:42.851499 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863ae34-7033-4c6c-8b8e-28d8f28a92c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:42.851967 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863ae34-7033-4c6c-8b8e-28d8f28a92c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:42.881813 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863ae34-7033-4c6c-8b8e-28d8f28a92c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:42.886347 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863ae34-7033-4c6c-8b8e-28d8f28a92c0 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:42.899102 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1723799-8dec-4cf4-a324-7db36b58299e None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:42.921118 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 735/1488] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:42.921400 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a35385-02e3-4080-b6b5-8892adaa7a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:42.922092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a35385-02e3-4080-b6b5-8892adaa7a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:42.922380 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a35385-02e3-4080-b6b5-8892adaa7a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:42.922684 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a35385-02e3-4080-b6b5-8892adaa7a21 admin admin] PATH_INFO: `/v3/projects/0175d72d8ca24f2b89ff29ea4d539148` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:42.929875 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a35385-02e3-4080-b6b5-8892adaa7a21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0175d72d8ca24f2b89ff29ea4d539148)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:42.940556 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a35385-02e3-4080-b6b5-8892adaa7a21 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:43.172472 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6e7b3-6133-48b0-8596-33c05297a91b None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:43.173075 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6e7b3-6133-48b0-8596-33c05297a91b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:43.173075 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6e7b3-6133-48b0-8596-33c05297a91b None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:43.175054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 736/1489] 10.222.0.36 () {54 vars in 921 bytes} [Sun Oct 30 00:48:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:43.188659 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-04a35385-02e3-4080-b6b5-8892adaa7a21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0175d72d8ca24f2b89ff29ea4d539148', 'request_id': 'req-04a35385-02e3-4080-b6b5-8892adaa7a21'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:43.201310 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde609f-b983-4bd2-a277-c1346037c48c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:43.201576 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde609f-b983-4bd2-a277-c1346037c48c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:43.201870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde609f-b983-4bd2-a277-c1346037c48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:43.210402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-04a35385-02e3-4080-b6b5-8892adaa7a21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0175d72d8ca24f2b89ff29ea4d539148', 'request_id': 'req-04a35385-02e3-4080-b6b5-8892adaa7a21'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:43.232837 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-04a35385-02e3-4080-b6b5-8892adaa7a21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0175d72d8ca24f2b89ff29ea4d539148', 'request_id': 'req-04a35385-02e3-4080-b6b5-8892adaa7a21'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:43.264259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-5cde609f-b983-4bd2-a277-c1346037c48c None None] MFA Rules not processed for user `adbba7be39a64230bab160ce77ce7586`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:43.273871 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-5cde609f-b983-4bd2-a277-c1346037c48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:43.277193 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-04a35385-02e3-4080-b6b5-8892adaa7a21 admin admin] The token cache is being invalidate because project 0175d72d8ca24f2b89ff29ea4d539148 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:48:43.277518 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-04a35385-02e3-4080-b6b5-8892adaa7a21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:43.279104 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-04a35385-02e3-4080-b6b5-8892adaa7a21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:43.281393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 754/1490] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:48:42 2022] DELETE /identity/v3/projects/0175d72d8ca24f2b89ff29ea4d539148 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:43.285862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a35385-02e3-4080-b6b5-8892adaa7a21 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:43.296319 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-04a35385-02e3-4080-b6b5-8892adaa7a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:43.558561 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a35385-02e3-4080-b6b5-8892adaa7a21 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:43.565535 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 737/1491] 10.222.0.36 () {58 vars in 1018 bytes} [Sun Oct 30 00:48:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:48:43.566618 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793e08d-8513-4144-bec0-275fd5a07d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:43.567699 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793e08d-8513-4144-bec0-275fd5a07d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:43.568323 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793e08d-8513-4144-bec0-275fd5a07d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:43.569075 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793e08d-8513-4144-bec0-275fd5a07d14 admin admin] PATH_INFO: `/v3/users/158114d85ff64e518778b409bd7e5691` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:43.570827 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793e08d-8513-4144-bec0-275fd5a07d14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=158114d85ff64e518778b409bd7e5691)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:43.574650 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793e08d-8513-4144-bec0-275fd5a07d14 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:43.577562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde609f-b983-4bd2-a277-c1346037c48c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:43.582865 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-5cde609f-b983-4bd2-a277-c1346037c48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:43.608305 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:43.652428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3793e08d-8513-4144-bec0-275fd5a07d14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '158114d85ff64e518778b409bd7e5691'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:43.653125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3793e08d-8513-4144-bec0-275fd5a07d14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '158114d85ff64e518778b409bd7e5691'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:43.671625 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde609f-b983-4bd2-a277-c1346037c48c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:43.683045 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3793e08d-8513-4144-bec0-275fd5a07d14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '158114d85ff64e518778b409bd7e5691'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:43.695396 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3793e08d-8513-4144-bec0-275fd5a07d14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '158114d85ff64e518778b409bd7e5691'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:43.709784 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3793e08d-8513-4144-bec0-275fd5a07d14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '158114d85ff64e518778b409bd7e5691'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:43.711296 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08886c9-0993-4da9-b8b0-1fbef78329c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:43.711984 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08886c9-0993-4da9-b8b0-1fbef78329c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:43.717279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08886c9-0993-4da9-b8b0-1fbef78329c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:43.717638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08886c9-0993-4da9-b8b0-1fbef78329c4 admin admin] PATH_INFO: `/v3/projects/240fafb59a9446c784beae25f92c532d` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:43.719320 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 755/1492] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:48:43 2022] DELETE /identity/v3/users/158114d85ff64e518778b409bd7e5691 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:43.726794 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08886c9-0993-4da9-b8b0-1fbef78329c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=240fafb59a9446c784beae25f92c532d)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:43.737990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08886c9-0993-4da9-b8b0-1fbef78329c4 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:43.965084 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b08886c9-0993-4da9-b8b0-1fbef78329c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '240fafb59a9446c784beae25f92c532d', 'request_id': 'req-b08886c9-0993-4da9-b8b0-1fbef78329c4'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:43.987226 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b08886c9-0993-4da9-b8b0-1fbef78329c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '240fafb59a9446c784beae25f92c532d', 'request_id': 'req-b08886c9-0993-4da9-b8b0-1fbef78329c4'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:43.998863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b08886c9-0993-4da9-b8b0-1fbef78329c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '240fafb59a9446c784beae25f92c532d', 'request_id': 'req-b08886c9-0993-4da9-b8b0-1fbef78329c4'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:44.030950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b08886c9-0993-4da9-b8b0-1fbef78329c4 admin admin] The token cache is being invalidate because project 240fafb59a9446c784beae25f92c532d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:48:44.031276 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b08886c9-0993-4da9-b8b0-1fbef78329c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:44.032596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b08886c9-0993-4da9-b8b0-1fbef78329c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:44.036050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 738/1493] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:48:43 2022] DELETE /identity/v3/projects/240fafb59a9446c784beae25f92c532d => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:44.040206 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793e08d-8513-4144-bec0-275fd5a07d14 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:44.041905 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-3793e08d-8513-4144-bec0-275fd5a07d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:44.362055 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793e08d-8513-4144-bec0-275fd5a07d14 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:44.518818 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08886c9-0993-4da9-b8b0-1fbef78329c4 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:44.520457 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b08886c9-0993-4da9-b8b0-1fbef78329c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:44.588226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae929d31-10e7-4baf-ba21-f9e72ef3e2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:44.588226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae929d31-10e7-4baf-ba21-f9e72ef3e2f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:44.588226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae929d31-10e7-4baf-ba21-f9e72ef3e2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:44.588226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae929d31-10e7-4baf-ba21-f9e72ef3e2f4 admin admin] PATH_INFO: `/v3/users/4cfb0decbe404e19a98555ca5b9cf636` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:44.588993 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae929d31-10e7-4baf-ba21-f9e72ef3e2f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cfb0decbe404e19a98555ca5b9cf636)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:44.592415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae929d31-10e7-4baf-ba21-f9e72ef3e2f4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:44.597804 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:44.696153 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:44.701300 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ae929d31-10e7-4baf-ba21-f9e72ef3e2f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cfb0decbe404e19a98555ca5b9cf636'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:44.702832 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ae929d31-10e7-4baf-ba21-f9e72ef3e2f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cfb0decbe404e19a98555ca5b9cf636'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:44.704762 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08886c9-0993-4da9-b8b0-1fbef78329c4 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:44.724318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c082b3-4540-4cdf-86a9-cd734e92b10b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:44.724318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c082b3-4540-4cdf-86a9-cd734e92b10b service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:44.724318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c082b3-4540-4cdf-86a9-cd734e92b10b service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:44.724318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c082b3-4540-4cdf-86a9-cd734e92b10b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:44.724724 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a0c082b3-4540-4cdf-86a9-cd734e92b10b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:44.733735 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ae929d31-10e7-4baf-ba21-f9e72ef3e2f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cfb0decbe404e19a98555ca5b9cf636'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:44.737943 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c082b3-4540-4cdf-86a9-cd734e92b10b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:44.743406 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ae929d31-10e7-4baf-ba21-f9e72ef3e2f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cfb0decbe404e19a98555ca5b9cf636'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:44.747888 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c082b3-4540-4cdf-86a9-cd734e92b10b service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:44.763355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 739/1494] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:48:44.767547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ae929d31-10e7-4baf-ba21-f9e72ef3e2f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cfb0decbe404e19a98555ca5b9cf636'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:44.821083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 756/1495] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:48:44 2022] DELETE /identity/v3/users/4cfb0decbe404e19a98555ca5b9cf636 => generated 0 bytes in 763 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:46.296720 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c082b3-4540-4cdf-86a9-cd734e92b10b service nova] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:46.298340 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a0c082b3-4540-4cdf-86a9-cd734e92b10b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:46.317449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c082b3-4540-4cdf-86a9-cd734e92b10b service nova] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:46.395063 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae929d31-10e7-4baf-ba21-f9e72ef3e2f4 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:46.396400 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ae929d31-10e7-4baf-ba21-f9e72ef3e2f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:46.421535 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e74ab4-83ea-446e-84b1-c0d353bbd8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:46.422745 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e74ab4-83ea-446e-84b1-c0d353bbd8f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:46.423177 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e74ab4-83ea-446e-84b1-c0d353bbd8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:46.423612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e74ab4-83ea-446e-84b1-c0d353bbd8f4 admin admin] PATH_INFO: `/v3/projects/5fbee574177440a7a2245f756d84f845` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:46.425045 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e74ab4-83ea-446e-84b1-c0d353bbd8f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fbee574177440a7a2245f756d84f845)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:46.428425 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e74ab4-83ea-446e-84b1-c0d353bbd8f4 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:46.566148 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-20e74ab4-83ea-446e-84b1-c0d353bbd8f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fbee574177440a7a2245f756d84f845', 'request_id': 'req-20e74ab4-83ea-446e-84b1-c0d353bbd8f4'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:46.583443 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-20e74ab4-83ea-446e-84b1-c0d353bbd8f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fbee574177440a7a2245f756d84f845', 'request_id': 'req-20e74ab4-83ea-446e-84b1-c0d353bbd8f4'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:46.594936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-20e74ab4-83ea-446e-84b1-c0d353bbd8f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fbee574177440a7a2245f756d84f845', 'request_id': 'req-20e74ab4-83ea-446e-84b1-c0d353bbd8f4'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:46.603382 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae929d31-10e7-4baf-ba21-f9e72ef3e2f4 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:46.617190 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-20e74ab4-83ea-446e-84b1-c0d353bbd8f4 admin admin] The token cache is being invalidate because project 5fbee574177440a7a2245f756d84f845 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:48:46.617551 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-20e74ab4-83ea-446e-84b1-c0d353bbd8f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:46.618347 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-20e74ab4-83ea-446e-84b1-c0d353bbd8f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:46.620630 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 740/1496] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:48:46 2022] DELETE /identity/v3/projects/5fbee574177440a7a2245f756d84f845 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:46.624598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e74ab4-83ea-446e-84b1-c0d353bbd8f4 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:46.626401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-20e74ab4-83ea-446e-84b1-c0d353bbd8f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:46.733650 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea6237-24b0-434f-8be2-87f2ce088bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:46.734596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea6237-24b0-434f-8be2-87f2ce088bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:46.734882 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea6237-24b0-434f-8be2-87f2ce088bab admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:46.735373 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea6237-24b0-434f-8be2-87f2ce088bab admin admin] PATH_INFO: `/v3/projects/7db9fa19df73499d843c7b35034aa7a7` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:46.736694 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea6237-24b0-434f-8be2-87f2ce088bab admin admin] RBAC: Authorizing `identity:delete_project(project_id=7db9fa19df73499d843c7b35034aa7a7)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:46.737124 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:46.740914 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea6237-24b0-434f-8be2-87f2ce088bab admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:46.789141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e74ab4-83ea-446e-84b1-c0d353bbd8f4 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:46.807507 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f9497-f584-40b5-8bcf-174361d5d182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:46.808307 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f9497-f584-40b5-8bcf-174361d5d182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:46.810389 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f9497-f584-40b5-8bcf-174361d5d182 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:46.811788 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f9497-f584-40b5-8bcf-174361d5d182 admin admin] PATH_INFO: `/v3/users/eafa63c27141480bb35a424ce44ae880` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:46.812702 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f9497-f584-40b5-8bcf-174361d5d182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eafa63c27141480bb35a424ce44ae880)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:46.816688 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f9497-f584-40b5-8bcf-174361d5d182 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:46.845577 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-caea6237-24b0-434f-8be2-87f2ce088bab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7db9fa19df73499d843c7b35034aa7a7', 'request_id': 'req-caea6237-24b0-434f-8be2-87f2ce088bab'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:46.873249 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-caea6237-24b0-434f-8be2-87f2ce088bab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7db9fa19df73499d843c7b35034aa7a7', 'request_id': 'req-caea6237-24b0-434f-8be2-87f2ce088bab'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:46.891324 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-caea6237-24b0-434f-8be2-87f2ce088bab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7db9fa19df73499d843c7b35034aa7a7', 'request_id': 'req-caea6237-24b0-434f-8be2-87f2ce088bab'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:46.929314 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:46.941544 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-caea6237-24b0-434f-8be2-87f2ce088bab admin admin] The token cache is being invalidate because project 7db9fa19df73499d843c7b35034aa7a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:48:46.942858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-caea6237-24b0-434f-8be2-87f2ce088bab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:46.946190 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-caea6237-24b0-434f-8be2-87f2ce088bab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:46.950356 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 757/1497] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:48:46 2022] DELETE /identity/v3/projects/7db9fa19df73499d843c7b35034aa7a7 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:46.955093 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e838c9-1e84-4fe7-9322-6b2cb348d176 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:46.955558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e838c9-1e84-4fe7-9322-6b2cb348d176 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:46.955932 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e838c9-1e84-4fe7-9322-6b2cb348d176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:46.962330 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e05f9497-f584-40b5-8bcf-174361d5d182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eafa63c27141480bb35a424ce44ae880'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:46.963210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e05f9497-f584-40b5-8bcf-174361d5d182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eafa63c27141480bb35a424ce44ae880'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:46.996868 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e05f9497-f584-40b5-8bcf-174361d5d182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eafa63c27141480bb35a424ce44ae880'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:47.009986 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e05f9497-f584-40b5-8bcf-174361d5d182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eafa63c27141480bb35a424ce44ae880'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:47.014680 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e05f9497-f584-40b5-8bcf-174361d5d182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eafa63c27141480bb35a424ce44ae880'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:47.040479 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-96e838c9-1e84-4fe7-9322-6b2cb348d176 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:47.041860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-96e838c9-1e84-4fe7-9322-6b2cb348d176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:47.047620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 741/1498] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:48:46 2022] DELETE /identity/v3/users/eafa63c27141480bb35a424ce44ae880 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:48:47.102036 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f9497-f584-40b5-8bcf-174361d5d182 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:47.103920 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e05f9497-f584-40b5-8bcf-174361d5d182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:47.266737 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f9497-f584-40b5-8bcf-174361d5d182 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:47.267652 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 758/1499] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:48:47.271198 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ab277-e396-4432-8bd6-977937335d7a None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:47.271805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ab277-e396-4432-8bd6-977937335d7a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:47.273031 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ab277-e396-4432-8bd6-977937335d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:47.278939 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ae0a1-cbb8-44a6-8e09-a34cc695eacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:47.280045 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ae0a1-cbb8-44a6-8e09-a34cc695eacd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:47.280575 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ae0a1-cbb8-44a6-8e09-a34cc695eacd admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:47.281029 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ae0a1-cbb8-44a6-8e09-a34cc695eacd admin admin] PATH_INFO: `/v3/projects/233b15d12d694eb88b15f9c30a61e105` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:47.283068 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ae0a1-cbb8-44a6-8e09-a34cc695eacd admin admin] RBAC: Authorizing `identity:delete_project(project_id=233b15d12d694eb88b15f9c30a61e105)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:47.288809 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ae0a1-cbb8-44a6-8e09-a34cc695eacd admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:47.302081 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-911ab277-e396-4432-8bd6-977937335d7a None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:47.308841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-911ab277-e396-4432-8bd6-977937335d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:47.331708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 759/1500] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:47.333796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ab277-e396-4432-8bd6-977937335d7a None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:47.367350 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ab277-e396-4432-8bd6-977937335d7a None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:47.410034 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24465d6-0df5-4e4e-befb-26e051928db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:47.412315 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24465d6-0df5-4e4e-befb-26e051928db5 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:47.416708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24465d6-0df5-4e4e-befb-26e051928db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:47.417142 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24465d6-0df5-4e4e-befb-26e051928db5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:47.427830 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24465d6-0df5-4e4e-befb-26e051928db5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:47.428269 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24465d6-0df5-4e4e-befb-26e051928db5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:47.429936 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24465d6-0df5-4e4e-befb-26e051928db5 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:47.448621 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-021ae0a1-cbb8-44a6-8e09-a34cc695eacd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '233b15d12d694eb88b15f9c30a61e105', 'request_id': 'req-021ae0a1-cbb8-44a6-8e09-a34cc695eacd'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:47.476637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 760/1501] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:47.478841 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-021ae0a1-cbb8-44a6-8e09-a34cc695eacd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '233b15d12d694eb88b15f9c30a61e105', 'request_id': 'req-021ae0a1-cbb8-44a6-8e09-a34cc695eacd'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:47.479695 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24465d6-0df5-4e4e-befb-26e051928db5 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:47.493086 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-021ae0a1-cbb8-44a6-8e09-a34cc695eacd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '233b15d12d694eb88b15f9c30a61e105', 'request_id': 'req-021ae0a1-cbb8-44a6-8e09-a34cc695eacd'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:47.502675 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24465d6-0df5-4e4e-befb-26e051928db5 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:47.518856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:47.537545 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-021ae0a1-cbb8-44a6-8e09-a34cc695eacd admin admin] The token cache is being invalidate because project 233b15d12d694eb88b15f9c30a61e105 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:48:47.537847 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-021ae0a1-cbb8-44a6-8e09-a34cc695eacd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:47.539515 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-021ae0a1-cbb8-44a6-8e09-a34cc695eacd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:47.546769 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 742/1502] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:48:47 2022] DELETE /identity/v3/projects/233b15d12d694eb88b15f9c30a61e105 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:47.550741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ae0a1-cbb8-44a6-8e09-a34cc695eacd admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:47.553706 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-021ae0a1-cbb8-44a6-8e09-a34cc695eacd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:47.636888 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc791599-846e-4edf-ab98-2732c116c1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:47.640054 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc791599-846e-4edf-ab98-2732c116c1ac admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:47.640304 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc791599-846e-4edf-ab98-2732c116c1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:47.641230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc791599-846e-4edf-ab98-2732c116c1ac admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:47.641230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc791599-846e-4edf-ab98-2732c116c1ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:47.645400 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc791599-846e-4edf-ab98-2732c116c1ac admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:47.694962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ae0a1-cbb8-44a6-8e09-a34cc695eacd admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:47.705750 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404d890-d1a4-438b-952a-aede06854fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:47.706542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404d890-d1a4-438b-952a-aede06854fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:47.706655 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404d890-d1a4-438b-952a-aede06854fca admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:47.707091 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404d890-d1a4-438b-952a-aede06854fca admin admin] PATH_INFO: `/v3/users/cb0d97d08f9746de97f2faa0281e5696` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:47.708434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404d890-d1a4-438b-952a-aede06854fca admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb0d97d08f9746de97f2faa0281e5696)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:47.710859 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404d890-d1a4-438b-952a-aede06854fca admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:47.718032 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 761/1503] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:47 2022] POST /identity/v3/projects => generated 375 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:47.719313 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc791599-846e-4edf-ab98-2732c116c1ac admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:47.722811 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-fc791599-846e-4edf-ab98-2732c116c1ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:47.830858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8404d890-d1a4-438b-952a-aede06854fca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb0d97d08f9746de97f2faa0281e5696'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:47.832257 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8404d890-d1a4-438b-952a-aede06854fca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb0d97d08f9746de97f2faa0281e5696'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:47.878736 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8404d890-d1a4-438b-952a-aede06854fca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb0d97d08f9746de97f2faa0281e5696'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:47.889260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8404d890-d1a4-438b-952a-aede06854fca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb0d97d08f9746de97f2faa0281e5696'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:47.900132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8404d890-d1a4-438b-952a-aede06854fca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb0d97d08f9746de97f2faa0281e5696'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:47.938577 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 743/1504] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:48:47 2022] DELETE /identity/v3/users/cb0d97d08f9746de97f2faa0281e5696 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:48:47.940611 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc791599-846e-4edf-ab98-2732c116c1ac admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:47.962978 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:48.034949 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead2da0-f33e-4b35-aa88-c77c9fe75063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:48.035873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead2da0-f33e-4b35-aa88-c77c9fe75063 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.036348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead2da0-f33e-4b35-aa88-c77c9fe75063 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.036454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead2da0-f33e-4b35-aa88-c77c9fe75063 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:48.038037 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead2da0-f33e-4b35-aa88-c77c9fe75063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:48.040037 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead2da0-f33e-4b35-aa88-c77c9fe75063 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:48.070371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 762/1505] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:47 2022] POST /identity/v3/users => generated 288 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:48:48.071618 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404d890-d1a4-438b-952a-aede06854fca admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:48.073212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-8404d890-d1a4-438b-952a-aede06854fca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:48.096489 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404d890-d1a4-438b-952a-aede06854fca admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:48.102970 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c534b-3b46-4d43-bdfa-e36025499bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:48.103702 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c534b-3b46-4d43-bdfa-e36025499bca admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.104066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c534b-3b46-4d43-bdfa-e36025499bca admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.105007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c534b-3b46-4d43-bdfa-e36025499bca admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:48.105744 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c534b-3b46-4d43-bdfa-e36025499bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:48.108772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c534b-3b46-4d43-bdfa-e36025499bca admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:48.120670 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 744/1506] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:48.125171 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead2da0-f33e-4b35-aa88-c77c9fe75063 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:48.135650 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead2da0-f33e-4b35-aa88-c77c9fe75063 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:48.141592 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf7221-698c-4cfc-b302-3dfc8103ddc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:48.142287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf7221-698c-4cfc-b302-3dfc8103ddc5 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.142669 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf7221-698c-4cfc-b302-3dfc8103ddc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.143342 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf7221-698c-4cfc-b302-3dfc8103ddc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:48.143997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf7221-698c-4cfc-b302-3dfc8103ddc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:48.146178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf7221-698c-4cfc-b302-3dfc8103ddc5 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:48.159873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 763/1507] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:48.164341 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c534b-3b46-4d43-bdfa-e36025499bca admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:48.177142 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c534b-3b46-4d43-bdfa-e36025499bca admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:48.183455 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74205a5c-e4b4-423d-a239-e63ca0181e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:48.184166 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74205a5c-e4b4-423d-a239-e63ca0181e94 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.184436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74205a5c-e4b4-423d-a239-e63ca0181e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.185338 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74205a5c-e4b4-423d-a239-e63ca0181e94 admin admin] PATH_INFO: `/v3/projects/19f20f48a5fc45aa829880ed1d7950d0/users/cfe7374737a04042904d4a87469046c8/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:48.205328 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74205a5c-e4b4-423d-a239-e63ca0181e94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f20f48a5fc45aa829880ed1d7950d0, user_id=cfe7374737a04042904d4a87469046c8, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:48.208886 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74205a5c-e4b4-423d-a239-e63ca0181e94 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:48.231620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 745/1508] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:48 2022] PUT /identity/v3/projects/19f20f48a5fc45aa829880ed1d7950d0/users/cfe7374737a04042904d4a87469046c8/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:48.243777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4c06f-3588-4e26-96a4-7db68fac0047 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.244121 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4c06f-3588-4e26-96a4-7db68fac0047 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.244522 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4c06f-3588-4e26-96a4-7db68fac0047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:48.280676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-b5a4c06f-3588-4e26-96a4-7db68fac0047 None None] MFA Rules not processed for user `cfe7374737a04042904d4a87469046c8`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:48.284178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-b5a4c06f-3588-4e26-96a4-7db68fac0047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:48.409147 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 764/1509] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:48.413707 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74205a5c-e4b4-423d-a239-e63ca0181e94 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:48.480645 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:48.531252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74205a5c-e4b4-423d-a239-e63ca0181e94 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:48.562083 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a5650-0d85-47d9-810a-0219768dacc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:48.565221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a5650-0d85-47d9-810a-0219768dacc8 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.565221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a5650-0d85-47d9-810a-0219768dacc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.565221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a5650-0d85-47d9-810a-0219768dacc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:48.568811 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a5650-0d85-47d9-810a-0219768dacc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:48.571352 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a5650-0d85-47d9-810a-0219768dacc8 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:48.634868 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 746/1510] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:48 2022] POST /identity/v3/projects => generated 377 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:48.636413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a4c06f-3588-4e26-96a4-7db68fac0047 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:48.651701 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a4c06f-3588-4e26-96a4-7db68fac0047 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:48.753847 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0f481-0d43-423f-8323-cea1959a95db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:48.754474 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0f481-0d43-423f-8323-cea1959a95db admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.754988 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0f481-0d43-423f-8323-cea1959a95db admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.754988 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0f481-0d43-423f-8323-cea1959a95db admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:48.755636 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f0f481-0d43-423f-8323-cea1959a95db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:48.757633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f0f481-0d43-423f-8323-cea1959a95db admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:48.791180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a5650-0d85-47d9-810a-0219768dacc8 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:48.794983 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 765/1511] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:48 2022] POST /identity/v3/users => generated 295 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:48.812138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a5650-0d85-47d9-810a-0219768dacc8 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:48.821422 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c87e9-0e83-480e-8e2e-8e9cf8770b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:48.821969 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c87e9-0e83-480e-8e2e-8e9cf8770b02 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.822234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c87e9-0e83-480e-8e2e-8e9cf8770b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.822505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c87e9-0e83-480e-8e2e-8e9cf8770b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:48.823151 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c87e9-0e83-480e-8e2e-8e9cf8770b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:48.825022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c87e9-0e83-480e-8e2e-8e9cf8770b02 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:48.862065 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 747/1512] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:48.865624 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0f481-0d43-423f-8323-cea1959a95db admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:48.880108 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0f481-0d43-423f-8323-cea1959a95db admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:48.890779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1369eb-a7b2-4c51-9ad8-2bc88a327623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:48.891677 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1369eb-a7b2-4c51-9ad8-2bc88a327623 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:48.891970 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1369eb-a7b2-4c51-9ad8-2bc88a327623 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:48.892390 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1369eb-a7b2-4c51-9ad8-2bc88a327623 admin admin] PATH_INFO: `/v3/projects/e42199fcb9a34be7a2bb49bd5c6d27e5/users/b189afa40b9d4e2097f31537cc8f9ae3/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:48.908282 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:48.921904 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1369eb-a7b2-4c51-9ad8-2bc88a327623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e42199fcb9a34be7a2bb49bd5c6d27e5, user_id=b189afa40b9d4e2097f31537cc8f9ae3, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:48.926805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1369eb-a7b2-4c51-9ad8-2bc88a327623 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:48.954345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 766/1513] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:48 2022] PUT /identity/v3/projects/e42199fcb9a34be7a2bb49bd5c6d27e5/users/b189afa40b9d4e2097f31537cc8f9ae3/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:48.958198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c87e9-0e83-480e-8e2e-8e9cf8770b02 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:48.974136 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c87e9-0e83-480e-8e2e-8e9cf8770b02 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:49.067751 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10f06d-39fb-4ccc-9f3d-5e524ff7ead7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:49.068620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10f06d-39fb-4ccc-9f3d-5e524ff7ead7 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:49.069080 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10f06d-39fb-4ccc-9f3d-5e524ff7ead7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:49.069504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10f06d-39fb-4ccc-9f3d-5e524ff7ead7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.070455 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be10f06d-39fb-4ccc-9f3d-5e524ff7ead7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:49.072596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be10f06d-39fb-4ccc-9f3d-5e524ff7ead7 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:49.090192 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1369eb-a7b2-4c51-9ad8-2bc88a327623 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:49.091874 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 748/1514] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:49.105166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1369eb-a7b2-4c51-9ad8-2bc88a327623 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:49.112129 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf73673-fbd8-414b-bc88-4735d633dc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:49.113217 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf73673-fbd8-414b-bc88-4735d633dc9b admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:49.113651 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf73673-fbd8-414b-bc88-4735d633dc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:49.114619 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf73673-fbd8-414b-bc88-4735d633dc9b admin admin] PATH_INFO: `/v3/projects/e42199fcb9a34be7a2bb49bd5c6d27e5/users/b189afa40b9d4e2097f31537cc8f9ae3/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.117128 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf73673-fbd8-414b-bc88-4735d633dc9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e42199fcb9a34be7a2bb49bd5c6d27e5, user_id=b189afa40b9d4e2097f31537cc8f9ae3, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:49.120365 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf73673-fbd8-414b-bc88-4735d633dc9b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:49.140391 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 767/1515] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:49 2022] PUT /identity/v3/projects/e42199fcb9a34be7a2bb49bd5c6d27e5/users/b189afa40b9d4e2097f31537cc8f9ae3/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:49.148313 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2eced-d933-4862-ba60-ef9434dba57f None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:49.148928 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2eced-d933-4862-ba60-ef9434dba57f None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:49.148928 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2eced-d933-4862-ba60-ef9434dba57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.189799 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-cbc2eced-d933-4862-ba60-ef9434dba57f None None] MFA Rules not processed for user `b189afa40b9d4e2097f31537cc8f9ae3`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:49.191540 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-cbc2eced-d933-4862-ba60-ef9434dba57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:49.203503 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf73673-fbd8-414b-bc88-4735d633dc9b admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:49.208933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-0cf73673-fbd8-414b-bc88-4735d633dc9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:49.338044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 749/1516] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:48:49.338977 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2eced-d933-4862-ba60-ef9434dba57f None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:49.340602 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-cbc2eced-d933-4862-ba60-ef9434dba57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:49.373005 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf73673-fbd8-414b-bc88-4735d633dc9b admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:49.378279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86a995-92eb-4be2-8706-d52a4ff63128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:49.378948 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86a995-92eb-4be2-8706-d52a4ff63128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:49.379292 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86a995-92eb-4be2-8706-d52a4ff63128 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:49.379638 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86a995-92eb-4be2-8706-d52a4ff63128 admin admin] PATH_INFO: `/v3/projects/cb47324846064bcdba1d823141034334` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.380902 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86a995-92eb-4be2-8706-d52a4ff63128 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb47324846064bcdba1d823141034334)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:49.382477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:49.383022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86a995-92eb-4be2-8706-d52a4ff63128 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:49.420728 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2eced-d933-4862-ba60-ef9434dba57f None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:49.426275 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7105d7-9bc6-4889-a573-2cf6df46561b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:49.427078 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7105d7-9bc6-4889-a573-2cf6df46561b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:49.427618 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7105d7-9bc6-4889-a573-2cf6df46561b admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:49.428005 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7105d7-9bc6-4889-a573-2cf6df46561b admin admin] PATH_INFO: `/v3/projects/bf980aeaf59b4c96ab489c41cdbebe25` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.433910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7105d7-9bc6-4889-a573-2cf6df46561b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf980aeaf59b4c96ab489c41cdbebe25)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:49.435631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7105d7-9bc6-4889-a573-2cf6df46561b admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:49.440565 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ff86a995-92eb-4be2-8706-d52a4ff63128 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb47324846064bcdba1d823141034334', 'request_id': 'req-ff86a995-92eb-4be2-8706-d52a4ff63128'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:49.449722 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ff86a995-92eb-4be2-8706-d52a4ff63128 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb47324846064bcdba1d823141034334', 'request_id': 'req-ff86a995-92eb-4be2-8706-d52a4ff63128'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:49.461605 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ff86a995-92eb-4be2-8706-d52a4ff63128 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb47324846064bcdba1d823141034334', 'request_id': 'req-ff86a995-92eb-4be2-8706-d52a4ff63128'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:49.484017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ff86a995-92eb-4be2-8706-d52a4ff63128 admin admin] The token cache is being invalidate because project cb47324846064bcdba1d823141034334 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:48:49.484528 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ff86a995-92eb-4be2-8706-d52a4ff63128 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:49.485345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ff86a995-92eb-4be2-8706-d52a4ff63128 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:49.488990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 768/1517] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:48:49 2022] DELETE /identity/v3/projects/cb47324846064bcdba1d823141034334 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:49.489960 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86a995-92eb-4be2-8706-d52a4ff63128 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:49.491406 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ff86a995-92eb-4be2-8706-d52a4ff63128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:49.520746 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1f7105d7-9bc6-4889-a573-2cf6df46561b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf980aeaf59b4c96ab489c41cdbebe25', 'request_id': 'req-1f7105d7-9bc6-4889-a573-2cf6df46561b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:49.530755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1f7105d7-9bc6-4889-a573-2cf6df46561b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf980aeaf59b4c96ab489c41cdbebe25', 'request_id': 'req-1f7105d7-9bc6-4889-a573-2cf6df46561b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:49.537942 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1f7105d7-9bc6-4889-a573-2cf6df46561b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf980aeaf59b4c96ab489c41cdbebe25', 'request_id': 'req-1f7105d7-9bc6-4889-a573-2cf6df46561b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:49.552121 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1f7105d7-9bc6-4889-a573-2cf6df46561b admin admin] The token cache is being invalidate because project bf980aeaf59b4c96ab489c41cdbebe25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:48:49.552449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1f7105d7-9bc6-4889-a573-2cf6df46561b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:49.553131 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1f7105d7-9bc6-4889-a573-2cf6df46561b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:48:49.554956 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 750/1518] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:48:49 2022] DELETE /identity/v3/projects/bf980aeaf59b4c96ab489c41cdbebe25 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:49.556297 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ff9e9-c4f2-43e9-8ca2-22ddfefaaee7 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:49.556582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ff9e9-c4f2-43e9-8ca2-22ddfefaaee7 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:49.556857 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ff9e9-c4f2-43e9-8ca2-22ddfefaaee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.586404 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86a995-92eb-4be2-8706-d52a4ff63128 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:49.593614 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-840ff9e9-c4f2-43e9-8ca2-22ddfefaaee7 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:49.594875 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-840ff9e9-c4f2-43e9-8ca2-22ddfefaaee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:49.619928 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:49.675446 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d3373-0d0b-4350-9d87-6369bab0af70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:49.676191 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d3373-0d0b-4350-9d87-6369bab0af70 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:49.676561 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d3373-0d0b-4350-9d87-6369bab0af70 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:49.676970 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d3373-0d0b-4350-9d87-6369bab0af70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.678840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e56d3373-0d0b-4350-9d87-6369bab0af70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:49.682173 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 751/1519] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:49.683552 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62bcc24-f12b-46e9-889c-64acad1d9eaf None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:49.683846 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62bcc24-f12b-46e9-889c-64acad1d9eaf None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:49.684140 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62bcc24-f12b-46e9-889c-64acad1d9eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.776211 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-c62bcc24-f12b-46e9-889c-64acad1d9eaf None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:49.776211 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c62bcc24-f12b-46e9-889c-64acad1d9eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:49.793286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 752/1520] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:49.797781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7d094-ba69-4f93-a269-5fc02a3fc828 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:49.798075 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7d094-ba69-4f93-a269-5fc02a3fc828 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:49.798354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7d094-ba69-4f93-a269-5fc02a3fc828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.805361 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56d3373-0d0b-4350-9d87-6369bab0af70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:49.806919 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56d3373-0d0b-4350-9d87-6369bab0af70 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:49.823313 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 769/1521] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:49.826919 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03cbb2-b3c4-4e92-b446-04ed4c3167c9 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:49.827224 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03cbb2-b3c4-4e92-b446-04ed4c3167c9 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:49.827561 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03cbb2-b3c4-4e92-b446-04ed4c3167c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.843241 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-44f7d094-ba69-4f93-a269-5fc02a3fc828 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:49.844590 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-44f7d094-ba69-4f93-a269-5fc02a3fc828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:49.854869 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 753/1522] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:49.859837 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7d094-ba69-4f93-a269-5fc02a3fc828 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:49.879752 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7d094-ba69-4f93-a269-5fc02a3fc828 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:49.885244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-4f03cbb2-b3c4-4e92-b446-04ed4c3167c9 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:49.887242 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4f03cbb2-b3c4-4e92-b446-04ed4c3167c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:49.897555 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b99c8-d198-42d8-8834-3069e0adeee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:49.898266 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b99c8-d198-42d8-8834-3069e0adeee1 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:49.898550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b99c8-d198-42d8-8834-3069e0adeee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:49.898847 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b99c8-d198-42d8-8834-3069e0adeee1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.899481 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b99c8-d198-42d8-8834-3069e0adeee1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:49.899781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b99c8-d198-42d8-8834-3069e0adeee1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:49.902084 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b99c8-d198-42d8-8834-3069e0adeee1 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:49.911664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 770/1523] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:49.916280 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03cbb2-b3c4-4e92-b446-04ed4c3167c9 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:49.918702 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 754/1524] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:48:49.923625 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b99c8-d198-42d8-8834-3069e0adeee1 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:49.934179 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03cbb2-b3c4-4e92-b446-04ed4c3167c9 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:49.939685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b99c8-d198-42d8-8834-3069e0adeee1 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:49.940297 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e802b87-48d3-4aa8-ad5c-09ef54f02781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:49.941164 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e802b87-48d3-4aa8-ad5c-09ef54f02781 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:49.941468 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e802b87-48d3-4aa8-ad5c-09ef54f02781 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:49.941796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e802b87-48d3-4aa8-ad5c-09ef54f02781 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.942679 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e802b87-48d3-4aa8-ad5c-09ef54f02781 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:48:49.943143 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e802b87-48d3-4aa8-ad5c-09ef54f02781 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:49.945044 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e802b87-48d3-4aa8-ad5c-09ef54f02781 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:49.946262 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd938b-3e86-4b76-9e8b-c26bd458bfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:49.947385 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd938b-3e86-4b76-9e8b-c26bd458bfad admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:49.948645 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd938b-3e86-4b76-9e8b-c26bd458bfad admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:49.949131 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd938b-3e86-4b76-9e8b-c26bd458bfad admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.949915 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd938b-3e86-4b76-9e8b-c26bd458bfad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:49.954021 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd938b-3e86-4b76-9e8b-c26bd458bfad admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:49.963290 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 771/1525] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:48:49.964632 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e802b87-48d3-4aa8-ad5c-09ef54f02781 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:49.978652 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e802b87-48d3-4aa8-ad5c-09ef54f02781 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:49.984858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1fdf3-2dc3-4b72-916a-336cabacb101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:49.985511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1fdf3-2dc3-4b72-916a-336cabacb101 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:49.986015 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1fdf3-2dc3-4b72-916a-336cabacb101 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:49.987547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1fdf3-2dc3-4b72-916a-336cabacb101 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:49.988386 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1fdf3-2dc3-4b72-916a-336cabacb101 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:49.990760 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1fdf3-2dc3-4b72-916a-336cabacb101 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:50.034085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 772/1526] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:49 2022] POST /identity/v3/projects => generated 367 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:50.035327 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1fdf3-2dc3-4b72-916a-336cabacb101 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:50.040073 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 755/1527] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:49 2022] POST /identity/v3/projects => generated 401 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:50.052563 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd938b-3e86-4b76-9e8b-c26bd458bfad admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:50.091629 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:50.143082 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1fdf3-2dc3-4b72-916a-336cabacb101 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:50.164810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd938b-3e86-4b76-9e8b-c26bd458bfad admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:50.172458 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111977a1-a25a-405b-aeb7-46d54a3203bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:50.173129 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111977a1-a25a-405b-aeb7-46d54a3203bd admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.173408 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111977a1-a25a-405b-aeb7-46d54a3203bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.173680 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111977a1-a25a-405b-aeb7-46d54a3203bd admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.174363 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111977a1-a25a-405b-aeb7-46d54a3203bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:50.181608 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111977a1-a25a-405b-aeb7-46d54a3203bd admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:50.198879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:50.219126 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 756/1528] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:50 2022] POST /identity/v3/users => generated 301 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:50.224852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111977a1-a25a-405b-aeb7-46d54a3203bd admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:50.241492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111977a1-a25a-405b-aeb7-46d54a3203bd admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:50.243972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2665a-6e34-4254-a523-765752d053a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:50.244647 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2665a-6e34-4254-a523-765752d053a9 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.245190 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2665a-6e34-4254-a523-765752d053a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.245190 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2665a-6e34-4254-a523-765752d053a9 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.245824 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2665a-6e34-4254-a523-765752d053a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:50.248207 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2665a-6e34-4254-a523-765752d053a9 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:50.261605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2feab87-7429-431b-8b0e-a7e3621a8069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:50.264114 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2feab87-7429-431b-8b0e-a7e3621a8069 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.264114 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2feab87-7429-431b-8b0e-a7e3621a8069 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.264114 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2feab87-7429-431b-8b0e-a7e3621a8069 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.264114 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2feab87-7429-431b-8b0e-a7e3621a8069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:50.266735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2feab87-7429-431b-8b0e-a7e3621a8069 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:50.288963 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 757/1529] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:50.292046 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2feab87-7429-431b-8b0e-a7e3621a8069 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:50.303964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 773/1530] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:50 2022] POST /identity/v3/users => generated 284 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:50.306655 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2665a-6e34-4254-a523-765752d053a9 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:50.314099 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2feab87-7429-431b-8b0e-a7e3621a8069 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:50.323099 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dba68-ff9b-4375-ae13-1fe7496d5fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:50.324049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dba68-ff9b-4375-ae13-1fe7496d5fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.325713 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dba68-ff9b-4375-ae13-1fe7496d5fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.326076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dba68-ff9b-4375-ae13-1fe7496d5fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.326893 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dba68-ff9b-4375-ae13-1fe7496d5fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:50.328672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dba68-ff9b-4375-ae13-1fe7496d5fb0 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:50.331179 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2665a-6e34-4254-a523-765752d053a9 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:50.344937 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b789c-09a8-452c-852e-77b3402ba507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:50.345569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b789c-09a8-452c-852e-77b3402ba507 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.345847 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b789c-09a8-452c-852e-77b3402ba507 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.346116 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b789c-09a8-452c-852e-77b3402ba507 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.347067 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b789c-09a8-452c-852e-77b3402ba507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:50.348434 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b789c-09a8-452c-852e-77b3402ba507 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:50.350034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 758/1531] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:50.351095 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dba68-ff9b-4375-ae13-1fe7496d5fb0 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:50.371391 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 774/1532] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:50.374912 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dba68-ff9b-4375-ae13-1fe7496d5fb0 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:50.376364 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b789c-09a8-452c-852e-77b3402ba507 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:50.385069 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caedf3b-d560-4ced-b0f6-3d8bec97ee7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:50.386469 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b789c-09a8-452c-852e-77b3402ba507 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:50.388024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caedf3b-d560-4ced-b0f6-3d8bec97ee7a admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.388708 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caedf3b-d560-4ced-b0f6-3d8bec97ee7a admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.389340 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caedf3b-d560-4ced-b0f6-3d8bec97ee7a admin admin] PATH_INFO: `/v3/projects/b768b44396994dafa037d4ea7d460ed8/users/2b9a451d455a483b81583cff22fdfe13/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.392578 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583cd7d-6989-474d-83c5-8700cdc39669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:50.393184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583cd7d-6989-474d-83c5-8700cdc39669 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.393463 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583cd7d-6989-474d-83c5-8700cdc39669 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.393735 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583cd7d-6989-474d-83c5-8700cdc39669 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.394374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583cd7d-6989-474d-83c5-8700cdc39669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:50.396494 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583cd7d-6989-474d-83c5-8700cdc39669 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:50.412127 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 775/1533] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:50.414039 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583cd7d-6989-474d-83c5-8700cdc39669 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:50.416825 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caedf3b-d560-4ced-b0f6-3d8bec97ee7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b768b44396994dafa037d4ea7d460ed8, user_id=2b9a451d455a483b81583cff22fdfe13, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:50.420444 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caedf3b-d560-4ced-b0f6-3d8bec97ee7a admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:50.425875 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583cd7d-6989-474d-83c5-8700cdc39669 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:50.431708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841e46c-9412-4bf4-bf22-b75d5a0156af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:50.432422 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841e46c-9412-4bf4-bf22-b75d5a0156af admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.432788 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841e46c-9412-4bf4-bf22-b75d5a0156af admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.433157 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841e46c-9412-4bf4-bf22-b75d5a0156af admin admin] PATH_INFO: `/v3/projects/944de69115e6480eb29bc317a4e5e5b2/users/027c45e46200450992eabaed48e9e716/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.436433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 759/1534] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:50 2022] PUT /identity/v3/projects/b768b44396994dafa037d4ea7d460ed8/users/2b9a451d455a483b81583cff22fdfe13/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:50.447031 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e03f4-d3c5-474c-ad98-6e054f9c7507 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.447323 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e03f4-d3c5-474c-ad98-6e054f9c7507 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.447600 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e03f4-d3c5-474c-ad98-6e054f9c7507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.454468 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841e46c-9412-4bf4-bf22-b75d5a0156af admin admin] RBAC: Authorizing `identity:create_grant(project_id=944de69115e6480eb29bc317a4e5e5b2, user_id=027c45e46200450992eabaed48e9e716, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:50.458393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841e46c-9412-4bf4-bf22-b75d5a0156af admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:50.478126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 776/1535] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:50 2022] PUT /identity/v3/projects/944de69115e6480eb29bc317a4e5e5b2/users/027c45e46200450992eabaed48e9e716/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:48:50.484708 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-ec8e03f4-d3c5-474c-ad98-6e054f9c7507 None None] MFA Rules not processed for user `2b9a451d455a483b81583cff22fdfe13`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:50.485151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d1fcd-44a2-4bf3-970b-8da2c52561c8 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.485439 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d1fcd-44a2-4bf3-970b-8da2c52561c8 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.485710 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d1fcd-44a2-4bf3-970b-8da2c52561c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.485965 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-ec8e03f4-d3c5-474c-ad98-6e054f9c7507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:50.520840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-9d7d1fcd-44a2-4bf3-970b-8da2c52561c8 None None] MFA Rules not processed for user `027c45e46200450992eabaed48e9e716`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:50.522066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-9d7d1fcd-44a2-4bf3-970b-8da2c52561c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:50.573802 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 760/1536] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:50 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:48:50.578524 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e03f4-d3c5-474c-ad98-6e054f9c7507 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:50.608735 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 777/1537] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:50.617632 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d1fcd-44a2-4bf3-970b-8da2c52561c8 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:50.618153 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-9d7d1fcd-44a2-4bf3-970b-8da2c52561c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:50.644978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:50.664258 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e03f4-d3c5-474c-ad98-6e054f9c7507 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:50.672300 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4fe57-f2fb-489c-a06d-c0522560c480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:50.672921 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4fe57-f2fb-489c-a06d-c0522560c480 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.673201 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4fe57-f2fb-489c-a06d-c0522560c480 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.673471 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4fe57-f2fb-489c-a06d-c0522560c480 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.674381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4fe57-f2fb-489c-a06d-c0522560c480 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:50.676722 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4fe57-f2fb-489c-a06d-c0522560c480 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:50.718708 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 761/1538] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:48:50 2022] POST /identity/v3/projects => generated 401 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:48:50.724092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d1fcd-44a2-4bf3-970b-8da2c52561c8 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:50.730662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:50.739911 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4fe57-f2fb-489c-a06d-c0522560c480 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:50.793113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168d49b-0c1c-4166-a77b-2441d43a1b4c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cf9451f2ef7478589b4a4278cb03252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:50.793737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168d49b-0c1c-4166-a77b-2441d43a1b4c service glance] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.794596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168d49b-0c1c-4166-a77b-2441d43a1b4c service glance] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.794875 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168d49b-0c1c-4166-a77b-2441d43a1b4c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.819479 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4fe57-f2fb-489c-a06d-c0522560c480 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:50.826483 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b680f-15db-4878-84ac-e872461ec7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:50.827292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b680f-15db-4878-84ac-e872461ec7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.828317 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b680f-15db-4878-84ac-e872461ec7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.828317 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b680f-15db-4878-84ac-e872461ec7a9 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.828619 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b680f-15db-4878-84ac-e872461ec7a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:50.830592 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b680f-15db-4878-84ac-e872461ec7a9 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:50.855007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 762/1539] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:48:50 2022] POST /identity/v3/users => generated 307 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:48:50.859948 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b680f-15db-4878-84ac-e872461ec7a9 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:50.862697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168d49b-0c1c-4166-a77b-2441d43a1b4c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:50.864255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168d49b-0c1c-4166-a77b-2441d43a1b4c service glance] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:50.870583 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b680f-15db-4878-84ac-e872461ec7a9 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:50.875329 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 778/1540] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:48:50.876546 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1daae-1e20-4899-bfe2-f17ddfd38af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:50.877242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1daae-1e20-4899-bfe2-f17ddfd38af0 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.877611 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1daae-1e20-4899-bfe2-f17ddfd38af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.877982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1daae-1e20-4899-bfe2-f17ddfd38af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.878692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f1daae-1e20-4899-bfe2-f17ddfd38af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:50.880448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f1daae-1e20-4899-bfe2-f17ddfd38af0 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:50.891930 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 763/1541] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:48:50.897296 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168d49b-0c1c-4166-a77b-2441d43a1b4c service glance] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:50.911094 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168d49b-0c1c-4166-a77b-2441d43a1b4c service glance] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:50.915802 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1380ea-9818-42f2-96df-6d56fa9a1764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:50.916514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1380ea-9818-42f2-96df-6d56fa9a1764 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.916792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1380ea-9818-42f2-96df-6d56fa9a1764 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.917068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1380ea-9818-42f2-96df-6d56fa9a1764 admin admin] PATH_INFO: `/v3/projects/e7bc304b908647fb8e071f37b8a0b18a/users/18f5dca6b7c74d6d96a8d2710f92f580/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.938287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1380ea-9818-42f2-96df-6d56fa9a1764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7bc304b908647fb8e071f37b8a0b18a, user_id=18f5dca6b7c74d6d96a8d2710f92f580, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:50.941321 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1380ea-9818-42f2-96df-6d56fa9a1764 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:50.950638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d5891-ced8-478c-80a4-9c0be321892f None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.951093 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d5891-ced8-478c-80a4-9c0be321892f None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.951468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d5891-ced8-478c-80a4-9c0be321892f None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:50.953736 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 764/1542] 10.222.0.36 () {54 vars in 921 bytes} [Sun Oct 30 00:48:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:48:50.957530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 779/1543] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:50 2022] PUT /identity/v3/projects/e7bc304b908647fb8e071f37b8a0b18a/users/18f5dca6b7c74d6d96a8d2710f92f580/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:50.961297 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d5891-ced8-478c-80a4-9c0be321892f None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:50.971515 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598138b3-2f57-4822-9e80-1a0cfb1f058a None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:50.971987 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598138b3-2f57-4822-9e80-1a0cfb1f058a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:50.972446 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598138b3-2f57-4822-9e80-1a0cfb1f058a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:51.002552 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-598138b3-2f57-4822-9e80-1a0cfb1f058a None None] MFA Rules not processed for user `adbba7be39a64230bab160ce77ce7586`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:51.003842 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-598138b3-2f57-4822-9e80-1a0cfb1f058a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:51.070925 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d5891-ced8-478c-80a4-9c0be321892f None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:51.077730 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14772435-6bf1-419f-b7f0-e3ab3e73b44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:51.078495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14772435-6bf1-419f-b7f0-e3ab3e73b44b admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:51.079282 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14772435-6bf1-419f-b7f0-e3ab3e73b44b admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:51.079770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14772435-6bf1-419f-b7f0-e3ab3e73b44b admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:51.080788 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14772435-6bf1-419f-b7f0-e3ab3e73b44b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:51.082659 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14772435-6bf1-419f-b7f0-e3ab3e73b44b admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:51.097326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 765/1544] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:48:51.101275 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14772435-6bf1-419f-b7f0-e3ab3e73b44b admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:51.117623 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14772435-6bf1-419f-b7f0-e3ab3e73b44b admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:51.118606 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 780/1545] 10.222.0.36 () {58 vars in 1018 bytes} [Sun Oct 30 00:48:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:48:51.129436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d43c6-2a3a-492a-8ad3-1da92c199d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:51.130481 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d43c6-2a3a-492a-8ad3-1da92c199d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:51.131120 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d43c6-2a3a-492a-8ad3-1da92c199d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:51.131553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d43c6-2a3a-492a-8ad3-1da92c199d03 admin admin] PATH_INFO: `/v3/projects/e7bc304b908647fb8e071f37b8a0b18a/users/18f5dca6b7c74d6d96a8d2710f92f580/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:51.134662 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d43c6-2a3a-492a-8ad3-1da92c199d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7bc304b908647fb8e071f37b8a0b18a, user_id=18f5dca6b7c74d6d96a8d2710f92f580, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:51.137600 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d43c6-2a3a-492a-8ad3-1da92c199d03 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:51.144236 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598138b3-2f57-4822-9e80-1a0cfb1f058a None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:51.145996 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-598138b3-2f57-4822-9e80-1a0cfb1f058a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:51.161775 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 766/1546] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:48:51 2022] PUT /identity/v3/projects/e7bc304b908647fb8e071f37b8a0b18a/users/18f5dca6b7c74d6d96a8d2710f92f580/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:48:51.175211 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9f6b5-dccd-4917-b63c-5b30f790a36d None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:51.175496 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9f6b5-dccd-4917-b63c-5b30f790a36d None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:51.175775 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9f6b5-dccd-4917-b63c-5b30f790a36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:51.183136 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598138b3-2f57-4822-9e80-1a0cfb1f058a None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:51.233239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-c8b9f6b5-dccd-4917-b63c-5b30f790a36d None None] MFA Rules not processed for user `18f5dca6b7c74d6d96a8d2710f92f580`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:48:51.236476 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c8b9f6b5-dccd-4917-b63c-5b30f790a36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:51.249928 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:51.267993 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:48:51.324969 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b149903-02f0-47a0-9579-74921f8b8cba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:51.326286 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b149903-02f0-47a0-9579-74921f8b8cba service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:51.326785 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b149903-02f0-47a0-9579-74921f8b8cba service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:51.327261 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b149903-02f0-47a0-9579-74921f8b8cba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:51.338390 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 767/1547] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:48:51 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:48:51.344915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b149903-02f0-47a0-9579-74921f8b8cba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:51.346935 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b149903-02f0-47a0-9579-74921f8b8cba service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:51.370490 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 781/1548] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:51.518793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9f6b5-dccd-4917-b63c-5b30f790a36d None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:51.520463 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c8b9f6b5-dccd-4917-b63c-5b30f790a36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:48:51.600579 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9f6b5-dccd-4917-b63c-5b30f790a36d None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:51.604993 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8525f-a878-49da-8380-cd5d4bdfe61f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:51.605633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8525f-a878-49da-8380-cd5d4bdfe61f service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:51.605955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8525f-a878-49da-8380-cd5d4bdfe61f service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:51.606235 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8525f-a878-49da-8380-cd5d4bdfe61f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:51.622903 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8525f-a878-49da-8380-cd5d4bdfe61f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:51.624870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8525f-a878-49da-8380-cd5d4bdfe61f service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:51.636646 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 768/1549] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:51 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:48:51.755558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b149903-02f0-47a0-9579-74921f8b8cba service nova] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:48:51.785243 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b149903-02f0-47a0-9579-74921f8b8cba service nova] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:48:51.790930 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6a106-2133-43f7-8d96-70421c7291b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:48:51.791600 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6a106-2133-43f7-8d96-70421c7291b5 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:48:51.791883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6a106-2133-43f7-8d96-70421c7291b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:48:51.792425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6a106-2133-43f7-8d96-70421c7291b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:48:51.875398 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6a106-2133-43f7-8d96-70421c7291b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:48:51.876796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6a106-2133-43f7-8d96-70421c7291b5 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:48:51.886680 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 782/1550] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:48:51 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:49:01.631454 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=44 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:01.881866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=48 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:02.137783 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8525f-a878-49da-8380-cd5d4bdfe61f service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:02.147767 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8525f-a878-49da-8380-cd5d4bdfe61f service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:02.152791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff9790-fd9f-450a-97b9-61bb16f50f93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:02.153628 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff9790-fd9f-450a-97b9-61bb16f50f93 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:02.156026 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff9790-fd9f-450a-97b9-61bb16f50f93 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:02.156491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff9790-fd9f-450a-97b9-61bb16f50f93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:02.158583 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-85ff9790-fd9f-450a-97b9-61bb16f50f93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:02.316124 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff9790-fd9f-450a-97b9-61bb16f50f93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:02.317617 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff9790-fd9f-450a-97b9-61bb16f50f93 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:02.339940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 769/1551] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:49:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:49:09.483235 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88d3bb-7257-426e-b356-6a431fcbf411 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:09.484011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88d3bb-7257-426e-b356-6a431fcbf411 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:09.484011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88d3bb-7257-426e-b356-6a431fcbf411 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:09.485379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 783/1552] 10.222.0.36 () {54 vars in 921 bytes} [Sun Oct 30 00:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:49:09.497915 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8195a7-0b8a-41b1-a301-a22bf19e6ffb None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:09.498490 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8195a7-0b8a-41b1-a301-a22bf19e6ffb None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:09.498490 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8195a7-0b8a-41b1-a301-a22bf19e6ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:09.745013 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-dc8195a7-0b8a-41b1-a301-a22bf19e6ffb None None] MFA Rules not processed for user `adbba7be39a64230bab160ce77ce7586`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:09.746259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-dc8195a7-0b8a-41b1-a301-a22bf19e6ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:09.763121 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 770/1553] 10.222.0.36 () {58 vars in 1018 bytes} [Sun Oct 30 00:49:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:49:09.775475 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88d3bb-7257-426e-b356-6a431fcbf411 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:09.789647 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88d3bb-7257-426e-b356-6a431fcbf411 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:09.795009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec25910-78c3-4818-9b32-b80b9aaee488 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:09.795603 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec25910-78c3-4818-9b32-b80b9aaee488 service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:09.796507 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec25910-78c3-4818-9b32-b80b9aaee488 service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:09.796744 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec25910-78c3-4818-9b32-b80b9aaee488 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:09.805314 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec25910-78c3-4818-9b32-b80b9aaee488 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:09.806943 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec25910-78c3-4818-9b32-b80b9aaee488 service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:09.817851 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 784/1554] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:49:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:49:16.429910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8195a7-0b8a-41b1-a301-a22bf19e6ffb None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:16.504788 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8195a7-0b8a-41b1-a301-a22bf19e6ffb None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:16.509679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff4013-c6dd-43ed-a2b0-c9ffd6e8fe79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:16.510352 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff4013-c6dd-43ed-a2b0-c9ffd6e8fe79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:16.510657 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff4013-c6dd-43ed-a2b0-c9ffd6e8fe79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:16.510932 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff4013-c6dd-43ed-a2b0-c9ffd6e8fe79 admin admin] PATH_INFO: `/v3/users/2b9a451d455a483b81583cff22fdfe13` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:16.512176 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff4013-c6dd-43ed-a2b0-c9ffd6e8fe79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b9a451d455a483b81583cff22fdfe13)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:16.514178 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff4013-c6dd-43ed-a2b0-c9ffd6e8fe79 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:16.574864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-7aff4013-c6dd-43ed-a2b0-c9ffd6e8fe79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b9a451d455a483b81583cff22fdfe13'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:16.575638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-7aff4013-c6dd-43ed-a2b0-c9ffd6e8fe79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b9a451d455a483b81583cff22fdfe13'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:16.594254 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-7aff4013-c6dd-43ed-a2b0-c9ffd6e8fe79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b9a451d455a483b81583cff22fdfe13'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:16.603800 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-7aff4013-c6dd-43ed-a2b0-c9ffd6e8fe79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b9a451d455a483b81583cff22fdfe13'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:16.604861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-7aff4013-c6dd-43ed-a2b0-c9ffd6e8fe79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b9a451d455a483b81583cff22fdfe13'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:16.624159 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 771/1555] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:49:16 2022] DELETE /identity/v3/users/2b9a451d455a483b81583cff22fdfe13 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:16.628709 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadffb56-d72c-43fb-98a1-e2cf7ece1508 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:16.629171 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadffb56-d72c-43fb-98a1-e2cf7ece1508 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:16.629292 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadffb56-d72c-43fb-98a1-e2cf7ece1508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:16.648479 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-aadffb56-d72c-43fb-98a1-e2cf7ece1508 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:16.649767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-aadffb56-d72c-43fb-98a1-e2cf7ece1508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:16.707702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 785/1556] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:49:16.717966 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff4013-c6dd-43ed-a2b0-c9ffd6e8fe79 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:16.719464 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-7aff4013-c6dd-43ed-a2b0-c9ffd6e8fe79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:16.729245 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:16.778280 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff4013-c6dd-43ed-a2b0-c9ffd6e8fe79 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:16.782547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dec6c9-3f28-4d25-b8be-6ff9e5862a32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:16.783153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dec6c9-3f28-4d25-b8be-6ff9e5862a32 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:16.783430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dec6c9-3f28-4d25-b8be-6ff9e5862a32 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:16.783699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dec6c9-3f28-4d25-b8be-6ff9e5862a32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:16.792981 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dec6c9-3f28-4d25-b8be-6ff9e5862a32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:16.794083 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dec6c9-3f28-4d25-b8be-6ff9e5862a32 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:16.804200 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 772/1557] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:49:17.093923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadffb56-d72c-43fb-98a1-e2cf7ece1508 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:17.095740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-aadffb56-d72c-43fb-98a1-e2cf7ece1508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:17.109448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadffb56-d72c-43fb-98a1-e2cf7ece1508 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:17.114226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5b4a7-cae0-4528-8576-f7d39612782c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:17.115027 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5b4a7-cae0-4528-8576-f7d39612782c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:17.115271 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5b4a7-cae0-4528-8576-f7d39612782c admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:17.115514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5b4a7-cae0-4528-8576-f7d39612782c admin admin] PATH_INFO: `/v3/projects/b768b44396994dafa037d4ea7d460ed8` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:17.117009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a5b4a7-cae0-4528-8576-f7d39612782c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b768b44396994dafa037d4ea7d460ed8)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:17.118933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a5b4a7-cae0-4528-8576-f7d39612782c admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:17.181282 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-57a5b4a7-cae0-4528-8576-f7d39612782c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b768b44396994dafa037d4ea7d460ed8', 'request_id': 'req-57a5b4a7-cae0-4528-8576-f7d39612782c'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:17.189676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-57a5b4a7-cae0-4528-8576-f7d39612782c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b768b44396994dafa037d4ea7d460ed8', 'request_id': 'req-57a5b4a7-cae0-4528-8576-f7d39612782c'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:17.197139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-57a5b4a7-cae0-4528-8576-f7d39612782c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b768b44396994dafa037d4ea7d460ed8', 'request_id': 'req-57a5b4a7-cae0-4528-8576-f7d39612782c'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:17.210017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-57a5b4a7-cae0-4528-8576-f7d39612782c admin admin] The token cache is being invalidate because project b768b44396994dafa037d4ea7d460ed8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:49:17.210304 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-57a5b4a7-cae0-4528-8576-f7d39612782c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:17.210959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-57a5b4a7-cae0-4528-8576-f7d39612782c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:17.212703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 786/1558] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:49:17 2022] DELETE /identity/v3/projects/b768b44396994dafa037d4ea7d460ed8 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:17.216621 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dec6c9-3f28-4d25-b8be-6ff9e5862a32 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:17.228530 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dec6c9-3f28-4d25-b8be-6ff9e5862a32 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:17.289386 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497778bb-bcab-4d40-951d-ef4285f31952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:17.289875 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497778bb-bcab-4d40-951d-ef4285f31952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:17.290173 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497778bb-bcab-4d40-951d-ef4285f31952 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:17.290452 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497778bb-bcab-4d40-951d-ef4285f31952 admin admin] PATH_INFO: `/v3/users/18f5dca6b7c74d6d96a8d2710f92f580` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:17.291612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497778bb-bcab-4d40-951d-ef4285f31952 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18f5dca6b7c74d6d96a8d2710f92f580)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:17.293610 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497778bb-bcab-4d40-951d-ef4285f31952 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:17.365910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-497778bb-bcab-4d40-951d-ef4285f31952 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18f5dca6b7c74d6d96a8d2710f92f580'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:17.366741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-497778bb-bcab-4d40-951d-ef4285f31952 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18f5dca6b7c74d6d96a8d2710f92f580'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:17.379630 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-497778bb-bcab-4d40-951d-ef4285f31952 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18f5dca6b7c74d6d96a8d2710f92f580'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:17.386391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-497778bb-bcab-4d40-951d-ef4285f31952 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18f5dca6b7c74d6d96a8d2710f92f580'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:17.387274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-497778bb-bcab-4d40-951d-ef4285f31952 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18f5dca6b7c74d6d96a8d2710f92f580'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:17.401123 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 773/1559] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:49:17 2022] DELETE /identity/v3/users/18f5dca6b7c74d6d96a8d2710f92f580 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:17.865633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5b4a7-cae0-4528-8576-f7d39612782c admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:17.868599 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-57a5b4a7-cae0-4528-8576-f7d39612782c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:17.886787 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5b4a7-cae0-4528-8576-f7d39612782c admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:17.947710 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe69c9-b155-4318-8a9a-84b1cb4eb9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:17.948375 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe69c9-b155-4318-8a9a-84b1cb4eb9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:17.948651 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe69c9-b155-4318-8a9a-84b1cb4eb9db admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:17.948921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe69c9-b155-4318-8a9a-84b1cb4eb9db admin admin] PATH_INFO: `/v3/projects/e7bc304b908647fb8e071f37b8a0b18a` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:17.950026 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe69c9-b155-4318-8a9a-84b1cb4eb9db admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7bc304b908647fb8e071f37b8a0b18a)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:17.952176 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe69c9-b155-4318-8a9a-84b1cb4eb9db admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:17.963523 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:17.999969 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-bfbe69c9-b155-4318-8a9a-84b1cb4eb9db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7bc304b908647fb8e071f37b8a0b18a', 'request_id': 'req-bfbe69c9-b155-4318-8a9a-84b1cb4eb9db'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:18.007034 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-bfbe69c9-b155-4318-8a9a-84b1cb4eb9db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7bc304b908647fb8e071f37b8a0b18a', 'request_id': 'req-bfbe69c9-b155-4318-8a9a-84b1cb4eb9db'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:18.013493 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-bfbe69c9-b155-4318-8a9a-84b1cb4eb9db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7bc304b908647fb8e071f37b8a0b18a', 'request_id': 'req-bfbe69c9-b155-4318-8a9a-84b1cb4eb9db'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:18.028159 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-bfbe69c9-b155-4318-8a9a-84b1cb4eb9db admin admin] The token cache is being invalidate because project e7bc304b908647fb8e071f37b8a0b18a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:49:18.028450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-bfbe69c9-b155-4318-8a9a-84b1cb4eb9db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:18.029118 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-bfbe69c9-b155-4318-8a9a-84b1cb4eb9db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:18.031411 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 787/1560] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:49:17 2022] DELETE /identity/v3/projects/e7bc304b908647fb8e071f37b8a0b18a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:18.036282 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8512b0-1ad0-4a99-ab10-e92ca2fa685f None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:18.036776 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8512b0-1ad0-4a99-ab10-e92ca2fa685f None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:18.036867 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8512b0-1ad0-4a99-ab10-e92ca2fa685f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:18.057597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-ea8512b0-1ad0-4a99-ab10-e92ca2fa685f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:18.058783 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-ea8512b0-1ad0-4a99-ab10-e92ca2fa685f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:18.083379 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:18.120915 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 774/1561] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:49:18.128956 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c1d43-bc59-4ed8-89a2-568a4dc36f7a None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:18.129317 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c1d43-bc59-4ed8-89a2-568a4dc36f7a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:18.129662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c1d43-bc59-4ed8-89a2-568a4dc36f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:18.149702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-814c1d43-bc59-4ed8-89a2-568a4dc36f7a None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:18.150921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-814c1d43-bc59-4ed8-89a2-568a4dc36f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:18.159704 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 788/1562] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:49:18.164104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8512b0-1ad0-4a99-ab10-e92ca2fa685f None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:18.174534 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8512b0-1ad0-4a99-ab10-e92ca2fa685f None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:18.179251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790aea4-ce09-48f0-92ae-2002d12a4c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:18.179855 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790aea4-ce09-48f0-92ae-2002d12a4c77 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:18.180176 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790aea4-ce09-48f0-92ae-2002d12a4c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:18.180446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790aea4-ce09-48f0-92ae-2002d12a4c77 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:18.181086 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790aea4-ce09-48f0-92ae-2002d12a4c77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:18.181382 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790aea4-ce09-48f0-92ae-2002d12a4c77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:18.182979 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790aea4-ce09-48f0-92ae-2002d12a4c77 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:18.194600 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 775/1563] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:18.198895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c1d43-bc59-4ed8-89a2-568a4dc36f7a None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:18.208428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c1d43-bc59-4ed8-89a2-568a4dc36f7a None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:18.213026 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461e8ec-495b-4b56-b93a-d42083f94b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:18.213619 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461e8ec-495b-4b56-b93a-d42083f94b5e admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:18.213902 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461e8ec-495b-4b56-b93a-d42083f94b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:18.214179 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461e8ec-495b-4b56-b93a-d42083f94b5e admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:18.214808 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461e8ec-495b-4b56-b93a-d42083f94b5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:18.216760 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461e8ec-495b-4b56-b93a-d42083f94b5e admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:18.245222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 789/1564] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:49:18 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:49:18.249916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790aea4-ce09-48f0-92ae-2002d12a4c77 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:18.259732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790aea4-ce09-48f0-92ae-2002d12a4c77 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:18.316514 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ef889-2ff1-400d-9393-8613168a14b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:18.317119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ef889-2ff1-400d-9393-8613168a14b9 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:18.317402 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ef889-2ff1-400d-9393-8613168a14b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:18.317678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ef889-2ff1-400d-9393-8613168a14b9 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:18.318299 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ef889-2ff1-400d-9393-8613168a14b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:18.320315 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ef889-2ff1-400d-9393-8613168a14b9 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:18.341314 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 776/1565] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:49:18 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:49:18.345906 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461e8ec-495b-4b56-b93a-d42083f94b5e admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:18.356643 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461e8ec-495b-4b56-b93a-d42083f94b5e admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:18.361857 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296d40e-a439-4842-801e-866c10bf809d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:18.362464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296d40e-a439-4842-801e-866c10bf809d admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:18.362746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296d40e-a439-4842-801e-866c10bf809d admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:18.363020 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296d40e-a439-4842-801e-866c10bf809d admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:18.363700 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296d40e-a439-4842-801e-866c10bf809d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:18.365338 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296d40e-a439-4842-801e-866c10bf809d admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:18.375738 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 790/1566] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:49:18.379876 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ef889-2ff1-400d-9393-8613168a14b9 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:18.389602 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ef889-2ff1-400d-9393-8613168a14b9 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:18.394504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790be747-c7d8-49b4-851a-88396c720eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:18.395105 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790be747-c7d8-49b4-851a-88396c720eff admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:18.395382 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790be747-c7d8-49b4-851a-88396c720eff admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:18.395659 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790be747-c7d8-49b4-851a-88396c720eff admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:18.396339 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790be747-c7d8-49b4-851a-88396c720eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:18.397958 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790be747-c7d8-49b4-851a-88396c720eff admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:18.413575 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 777/1567] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:49:18.417769 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296d40e-a439-4842-801e-866c10bf809d admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:18.428565 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296d40e-a439-4842-801e-866c10bf809d admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:18.433098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b1add-d749-43bb-a530-20e021662406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:18.433740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b1add-d749-43bb-a530-20e021662406 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:18.434040 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b1add-d749-43bb-a530-20e021662406 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:18.434303 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b1add-d749-43bb-a530-20e021662406 admin admin] PATH_INFO: `/v3/projects/9c74cb4be3e74ddc84759f9bb29cf47f/users/2b3ebf6b7b7d4cefbcfcbf84c47f363c/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:18.449269 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b1add-d749-43bb-a530-20e021662406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c74cb4be3e74ddc84759f9bb29cf47f, user_id=2b3ebf6b7b7d4cefbcfcbf84c47f363c, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:18.452546 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b1add-d749-43bb-a530-20e021662406 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:18.465915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 791/1568] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:49:18 2022] PUT /identity/v3/projects/9c74cb4be3e74ddc84759f9bb29cf47f/users/2b3ebf6b7b7d4cefbcfcbf84c47f363c/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:18.477748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55677ef0-4a8a-485f-99f7-d51eca26e79c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:18.478034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55677ef0-4a8a-485f-99f7-d51eca26e79c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:18.478309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55677ef0-4a8a-485f-99f7-d51eca26e79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:18.507212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-55677ef0-4a8a-485f-99f7-d51eca26e79c None None] MFA Rules not processed for user `2b3ebf6b7b7d4cefbcfcbf84c47f363c`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:18.508431 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-55677ef0-4a8a-485f-99f7-d51eca26e79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:18.579254 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 778/1569] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:49:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:49:18.583966 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b1add-d749-43bb-a530-20e021662406 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:18.648837 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b1add-d749-43bb-a530-20e021662406 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:18.654367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60b93a-aaaf-4854-bd9f-5234475824b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:18.655735 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60b93a-aaaf-4854-bd9f-5234475824b0 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:18.656050 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60b93a-aaaf-4854-bd9f-5234475824b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:18.656502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60b93a-aaaf-4854-bd9f-5234475824b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:18.656975 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f60b93a-aaaf-4854-bd9f-5234475824b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:18.658895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f60b93a-aaaf-4854-bd9f-5234475824b0 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:18.696117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 792/1570] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:49:18 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:49:18.699233 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55677ef0-4a8a-485f-99f7-d51eca26e79c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:18.710772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55677ef0-4a8a-485f-99f7-d51eca26e79c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:18.732175 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:18.779721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5a62b-ec76-438a-afba-5a151ca4ae34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:18.780564 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5a62b-ec76-438a-afba-5a151ca4ae34 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:18.780821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5a62b-ec76-438a-afba-5a151ca4ae34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:18.781097 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5a62b-ec76-438a-afba-5a151ca4ae34 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:18.781841 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5a62b-ec76-438a-afba-5a151ca4ae34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:18.783898 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5a62b-ec76-438a-afba-5a151ca4ae34 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:18.806922 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 779/1571] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:49:18 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:49:18.810838 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60b93a-aaaf-4854-bd9f-5234475824b0 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:18.821973 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60b93a-aaaf-4854-bd9f-5234475824b0 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:18.826528 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b7c78-bb8a-4a6d-abb4-e4c291638588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:18.827348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b7c78-bb8a-4a6d-abb4-e4c291638588 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:18.827622 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b7c78-bb8a-4a6d-abb4-e4c291638588 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:18.827895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b7c78-bb8a-4a6d-abb4-e4c291638588 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:18.828582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b7c78-bb8a-4a6d-abb4-e4c291638588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:18.830157 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b7c78-bb8a-4a6d-abb4-e4c291638588 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:18.840609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 793/1572] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:49:18.845943 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5a62b-ec76-438a-afba-5a151ca4ae34 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:18.862352 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5a62b-ec76-438a-afba-5a151ca4ae34 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:18.871351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8160551-cf09-4b14-a749-5e9e902c46b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:18.871351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8160551-cf09-4b14-a749-5e9e902c46b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:18.871351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8160551-cf09-4b14-a749-5e9e902c46b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:18.871642 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8160551-cf09-4b14-a749-5e9e902c46b3 admin admin] PATH_INFO: `/v3/projects/6f9a4dbc457c4e10b3c425dd01aff8ea/users/607f0ed46dc84207845aa86f36118546/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:18.898194 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8160551-cf09-4b14-a749-5e9e902c46b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f9a4dbc457c4e10b3c425dd01aff8ea, user_id=607f0ed46dc84207845aa86f36118546, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:18.903201 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8160551-cf09-4b14-a749-5e9e902c46b3 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:18.923995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 780/1573] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:49:18 2022] PUT /identity/v3/projects/6f9a4dbc457c4e10b3c425dd01aff8ea/users/607f0ed46dc84207845aa86f36118546/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:18.928207 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b7c78-bb8a-4a6d-abb4-e4c291638588 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:18.940964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b7c78-bb8a-4a6d-abb4-e4c291638588 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:18.949256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:19.003085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa022e-9b2e-4535-aede-b4bf520947b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:19.003706 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa022e-9b2e-4535-aede-b4bf520947b6 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:19.003989 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa022e-9b2e-4535-aede-b4bf520947b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:19.004310 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa022e-9b2e-4535-aede-b4bf520947b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:19.004959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa022e-9b2e-4535-aede-b4bf520947b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:19.006741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa022e-9b2e-4535-aede-b4bf520947b6 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:19.017665 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 794/1574] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:49:19.021778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8160551-cf09-4b14-a749-5e9e902c46b3 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:19.032568 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8160551-cf09-4b14-a749-5e9e902c46b3 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:19.037172 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a121a-b5da-4415-acdd-65158ab0b108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:19.037837 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a121a-b5da-4415-acdd-65158ab0b108 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:19.038119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a121a-b5da-4415-acdd-65158ab0b108 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:19.038400 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a121a-b5da-4415-acdd-65158ab0b108 admin admin] PATH_INFO: `/v3/projects/6f9a4dbc457c4e10b3c425dd01aff8ea/users/607f0ed46dc84207845aa86f36118546/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:19.040352 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a121a-b5da-4415-acdd-65158ab0b108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f9a4dbc457c4e10b3c425dd01aff8ea, user_id=607f0ed46dc84207845aa86f36118546, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:19.042993 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a121a-b5da-4415-acdd-65158ab0b108 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:19.054670 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 781/1575] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:49:19 2022] PUT /identity/v3/projects/6f9a4dbc457c4e10b3c425dd01aff8ea/users/607f0ed46dc84207845aa86f36118546/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:19.065158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fc94c-79be-47ec-8a20-2c96834bec85 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:19.065449 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fc94c-79be-47ec-8a20-2c96834bec85 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:19.065734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fc94c-79be-47ec-8a20-2c96834bec85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:19.095770 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-591fc94c-79be-47ec-8a20-2c96834bec85 None None] MFA Rules not processed for user `607f0ed46dc84207845aa86f36118546`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:19.097134 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-591fc94c-79be-47ec-8a20-2c96834bec85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:19.165802 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 795/1576] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:49:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:49:19.177318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a121a-b5da-4415-acdd-65158ab0b108 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:19.179167 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-dd1a121a-b5da-4415-acdd-65158ab0b108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:19.237572 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a121a-b5da-4415-acdd-65158ab0b108 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:19.241148 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def75b7-9a03-49a1-b639-912b0bc98724 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:19.241745 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def75b7-9a03-49a1-b639-912b0bc98724 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:19.242024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def75b7-9a03-49a1-b639-912b0bc98724 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:19.242295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def75b7-9a03-49a1-b639-912b0bc98724 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:19.251730 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def75b7-9a03-49a1-b639-912b0bc98724 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:19.253198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def75b7-9a03-49a1-b639-912b0bc98724 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:19.262443 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 782/1577] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:49:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:49:19.346673 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fc94c-79be-47ec-8a20-2c96834bec85 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:19.356291 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fc94c-79be-47ec-8a20-2c96834bec85 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:19.360448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf03db-717d-4953-97fc-2c6ca19b344d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:19.361042 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf03db-717d-4953-97fc-2c6ca19b344d service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:19.361321 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf03db-717d-4953-97fc-2c6ca19b344d service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:19.361597 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf03db-717d-4953-97fc-2c6ca19b344d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:19.417019 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf03db-717d-4953-97fc-2c6ca19b344d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:19.418369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf03db-717d-4953-97fc-2c6ca19b344d service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:19.428705 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 796/1578] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:49:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:49:29.258278 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=59,keystone:INSERT=5 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:29.424978 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=86 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:33.138445 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def75b7-9a03-49a1-b639-912b0bc98724 service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:33.225245 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def75b7-9a03-49a1-b639-912b0bc98724 service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:33.230619 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb324380-ebe3-4ae5-b300-d71ead29723b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:33.231258 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb324380-ebe3-4ae5-b300-d71ead29723b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:33.231545 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb324380-ebe3-4ae5-b300-d71ead29723b admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:33.231939 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb324380-ebe3-4ae5-b300-d71ead29723b admin admin] PATH_INFO: `/v3/users/2b3ebf6b7b7d4cefbcfcbf84c47f363c` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:33.233279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb324380-ebe3-4ae5-b300-d71ead29723b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b3ebf6b7b7d4cefbcfcbf84c47f363c)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:33.235423 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb324380-ebe3-4ae5-b300-d71ead29723b admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:33.301210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-cb324380-ebe3-4ae5-b300-d71ead29723b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b3ebf6b7b7d4cefbcfcbf84c47f363c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:33.302030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-cb324380-ebe3-4ae5-b300-d71ead29723b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b3ebf6b7b7d4cefbcfcbf84c47f363c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:33.319890 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-cb324380-ebe3-4ae5-b300-d71ead29723b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b3ebf6b7b7d4cefbcfcbf84c47f363c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:33.329228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-cb324380-ebe3-4ae5-b300-d71ead29723b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b3ebf6b7b7d4cefbcfcbf84c47f363c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:33.330272 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-cb324380-ebe3-4ae5-b300-d71ead29723b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b3ebf6b7b7d4cefbcfcbf84c47f363c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:33.346422 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 783/1579] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:49:33 2022] DELETE /identity/v3/users/2b3ebf6b7b7d4cefbcfcbf84c47f363c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:33.350936 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcd86d-c94d-48bf-9f6b-cd6a696fc281 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:33.351430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcd86d-c94d-48bf-9f6b-cd6a696fc281 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:33.351521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcd86d-c94d-48bf-9f6b-cd6a696fc281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:33.371303 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-3cfcd86d-c94d-48bf-9f6b-cd6a696fc281 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:33.372699 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-3cfcd86d-c94d-48bf-9f6b-cd6a696fc281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:33.439727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 797/1580] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:49:33.449604 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb324380-ebe3-4ae5-b300-d71ead29723b admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:33.451520 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-cb324380-ebe3-4ae5-b300-d71ead29723b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:33.521742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb324380-ebe3-4ae5-b300-d71ead29723b admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:33.525332 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201fa05-ce98-41ef-a438-4ca5af279d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:33.525966 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201fa05-ce98-41ef-a438-4ca5af279d46 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:33.526228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201fa05-ce98-41ef-a438-4ca5af279d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:33.526539 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201fa05-ce98-41ef-a438-4ca5af279d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:33.536831 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201fa05-ce98-41ef-a438-4ca5af279d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:33.538531 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201fa05-ce98-41ef-a438-4ca5af279d46 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:33.548843 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 784/1581] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:49:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:49:33.908198 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcd86d-c94d-48bf-9f6b-cd6a696fc281 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:33.909793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-3cfcd86d-c94d-48bf-9f6b-cd6a696fc281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:33.922788 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcd86d-c94d-48bf-9f6b-cd6a696fc281 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:33.928200 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b25cb-2858-4bf0-a729-fd4f44eb9248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:33.929107 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b25cb-2858-4bf0-a729-fd4f44eb9248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:33.929466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b25cb-2858-4bf0-a729-fd4f44eb9248 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:33.931091 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b25cb-2858-4bf0-a729-fd4f44eb9248 admin admin] PATH_INFO: `/v3/projects/9c74cb4be3e74ddc84759f9bb29cf47f` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:33.931312 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b25cb-2858-4bf0-a729-fd4f44eb9248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c74cb4be3e74ddc84759f9bb29cf47f)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:33.933283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b25cb-2858-4bf0-a729-fd4f44eb9248 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:33.992849 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-803b25cb-2858-4bf0-a729-fd4f44eb9248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c74cb4be3e74ddc84759f9bb29cf47f', 'request_id': 'req-803b25cb-2858-4bf0-a729-fd4f44eb9248'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:34.001474 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-803b25cb-2858-4bf0-a729-fd4f44eb9248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c74cb4be3e74ddc84759f9bb29cf47f', 'request_id': 'req-803b25cb-2858-4bf0-a729-fd4f44eb9248'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:34.007322 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-803b25cb-2858-4bf0-a729-fd4f44eb9248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c74cb4be3e74ddc84759f9bb29cf47f', 'request_id': 'req-803b25cb-2858-4bf0-a729-fd4f44eb9248'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:34.023465 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-803b25cb-2858-4bf0-a729-fd4f44eb9248 admin admin] The token cache is being invalidate because project 9c74cb4be3e74ddc84759f9bb29cf47f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:49:34.023811 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-803b25cb-2858-4bf0-a729-fd4f44eb9248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:34.024455 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-803b25cb-2858-4bf0-a729-fd4f44eb9248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:34.026036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 798/1582] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:49:33 2022] DELETE /identity/v3/projects/9c74cb4be3e74ddc84759f9bb29cf47f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:34.030196 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201fa05-ce98-41ef-a438-4ca5af279d46 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:34.039900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201fa05-ce98-41ef-a438-4ca5af279d46 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:34.060466 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:34.108478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec2424-6b8c-4460-bd45-e5478880b299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:34.109245 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec2424-6b8c-4460-bd45-e5478880b299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:34.109526 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec2424-6b8c-4460-bd45-e5478880b299 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:34.109897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec2424-6b8c-4460-bd45-e5478880b299 admin admin] PATH_INFO: `/v3/users/607f0ed46dc84207845aa86f36118546` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:34.111292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec2424-6b8c-4460-bd45-e5478880b299 admin admin] RBAC: Authorizing `identity:delete_user(user_id=607f0ed46dc84207845aa86f36118546)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:34.113415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec2424-6b8c-4460-bd45-e5478880b299 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:34.176245 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1eec2424-6b8c-4460-bd45-e5478880b299 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '607f0ed46dc84207845aa86f36118546'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:34.177449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1eec2424-6b8c-4460-bd45-e5478880b299 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '607f0ed46dc84207845aa86f36118546'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:34.200669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1eec2424-6b8c-4460-bd45-e5478880b299 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '607f0ed46dc84207845aa86f36118546'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:34.210912 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1eec2424-6b8c-4460-bd45-e5478880b299 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '607f0ed46dc84207845aa86f36118546'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:34.211628 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1eec2424-6b8c-4460-bd45-e5478880b299 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '607f0ed46dc84207845aa86f36118546'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:34.231819 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 785/1583] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:49:34 2022] DELETE /identity/v3/users/607f0ed46dc84207845aa86f36118546 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:34.512425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b25cb-2858-4bf0-a729-fd4f44eb9248 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:34.514430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-803b25cb-2858-4bf0-a729-fd4f44eb9248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:34.539281 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b25cb-2858-4bf0-a729-fd4f44eb9248 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:34.600804 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbbfa1a-2906-44a6-8437-ef381abc629c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:34.601442 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbfa1a-2906-44a6-8437-ef381abc629c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:34.601724 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbfa1a-2906-44a6-8437-ef381abc629c admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:34.601998 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbbfa1a-2906-44a6-8437-ef381abc629c admin admin] PATH_INFO: `/v3/projects/6f9a4dbc457c4e10b3c425dd01aff8ea` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:34.603115 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbbfa1a-2906-44a6-8437-ef381abc629c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f9a4dbc457c4e10b3c425dd01aff8ea)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:34.605071 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbbfa1a-2906-44a6-8437-ef381abc629c admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:34.634835 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:34.653661 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7dbbfa1a-2906-44a6-8437-ef381abc629c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f9a4dbc457c4e10b3c425dd01aff8ea', 'request_id': 'req-7dbbfa1a-2906-44a6-8437-ef381abc629c'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:34.660452 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7dbbfa1a-2906-44a6-8437-ef381abc629c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f9a4dbc457c4e10b3c425dd01aff8ea', 'request_id': 'req-7dbbfa1a-2906-44a6-8437-ef381abc629c'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:34.665919 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7dbbfa1a-2906-44a6-8437-ef381abc629c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f9a4dbc457c4e10b3c425dd01aff8ea', 'request_id': 'req-7dbbfa1a-2906-44a6-8437-ef381abc629c'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:34.676811 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7dbbfa1a-2906-44a6-8437-ef381abc629c admin admin] The token cache is being invalidate because project 6f9a4dbc457c4e10b3c425dd01aff8ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:49:34.677097 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7dbbfa1a-2906-44a6-8437-ef381abc629c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:34.677726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7dbbfa1a-2906-44a6-8437-ef381abc629c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:34.679196 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 799/1584] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:49:34 2022] DELETE /identity/v3/projects/6f9a4dbc457c4e10b3c425dd01aff8ea => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:34.686376 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c4796-4fae-42fe-bf48-b527add0f434 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:34.686742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c4796-4fae-42fe-bf48-b527add0f434 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:34.687044 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c4796-4fae-42fe-bf48-b527add0f434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:34.707362 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-a59c4796-4fae-42fe-bf48-b527add0f434 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:34.708610 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a59c4796-4fae-42fe-bf48-b527add0f434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:34.768960 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 786/1585] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:49:34.777953 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7169ff-18a3-4848-b00d-d15ace24700f None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:34.778236 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7169ff-18a3-4848-b00d-d15ace24700f None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:34.778511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7169ff-18a3-4848-b00d-d15ace24700f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:34.797925 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-2b7169ff-18a3-4848-b00d-d15ace24700f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:34.799149 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-2b7169ff-18a3-4848-b00d-d15ace24700f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:34.807989 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 800/1586] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:49:34.813166 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c4796-4fae-42fe-bf48-b527add0f434 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:34.823204 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c4796-4fae-42fe-bf48-b527add0f434 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:34.828515 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1bd519-43b6-4e36-aeb5-e2fd5e95e122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:34.829116 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bd519-43b6-4e36-aeb5-e2fd5e95e122 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:34.829400 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bd519-43b6-4e36-aeb5-e2fd5e95e122 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:34.829676 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bd519-43b6-4e36-aeb5-e2fd5e95e122 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:34.830358 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1bd519-43b6-4e36-aeb5-e2fd5e95e122 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:34.830659 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1bd519-43b6-4e36-aeb5-e2fd5e95e122 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:34.832280 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1bd519-43b6-4e36-aeb5-e2fd5e95e122 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:34.843759 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 787/1587] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:34.848090 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7169ff-18a3-4848-b00d-d15ace24700f None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:34.857243 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7169ff-18a3-4848-b00d-d15ace24700f None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:34.862330 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd576b-9679-4e77-b04f-0b0f7370ab6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:34.862912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd576b-9679-4e77-b04f-0b0f7370ab6d admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:34.863189 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd576b-9679-4e77-b04f-0b0f7370ab6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:34.863457 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd576b-9679-4e77-b04f-0b0f7370ab6d admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:34.864094 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd576b-9679-4e77-b04f-0b0f7370ab6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:34.866007 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd576b-9679-4e77-b04f-0b0f7370ab6d admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:34.899877 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1bd519-43b6-4e36-aeb5-e2fd5e95e122 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:34.904183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 801/1588] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:49:34 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:49:34.911558 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1bd519-43b6-4e36-aeb5-e2fd5e95e122 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:34.939718 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:34.977391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d93cb4-5e80-461a-949f-4a6358a1697f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:34.978008 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d93cb4-5e80-461a-949f-4a6358a1697f admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:34.978286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d93cb4-5e80-461a-949f-4a6358a1697f admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:34.978559 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d93cb4-5e80-461a-949f-4a6358a1697f admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:34.979215 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d93cb4-5e80-461a-949f-4a6358a1697f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:34.981174 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d93cb4-5e80-461a-949f-4a6358a1697f admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:35.004920 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 788/1589] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:49:34 2022] POST /identity/v3/users => generated 282 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:49:35.008971 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd576b-9679-4e77-b04f-0b0f7370ab6d admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:35.019864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd576b-9679-4e77-b04f-0b0f7370ab6d admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:35.024564 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2536cf0-955f-49bd-96c2-f2d61017a265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:35.025173 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2536cf0-955f-49bd-96c2-f2d61017a265 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:35.025460 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2536cf0-955f-49bd-96c2-f2d61017a265 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:35.025716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2536cf0-955f-49bd-96c2-f2d61017a265 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:35.026354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2536cf0-955f-49bd-96c2-f2d61017a265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:35.027948 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2536cf0-955f-49bd-96c2-f2d61017a265 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:35.039356 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 802/1590] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:49:35.043360 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d93cb4-5e80-461a-949f-4a6358a1697f admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:35.054454 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d93cb4-5e80-461a-949f-4a6358a1697f admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:35.059591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38056a11-ca37-45dc-abe2-42f4b169e932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:35.060224 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38056a11-ca37-45dc-abe2-42f4b169e932 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:35.060503 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38056a11-ca37-45dc-abe2-42f4b169e932 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:35.060775 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38056a11-ca37-45dc-abe2-42f4b169e932 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:35.062003 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38056a11-ca37-45dc-abe2-42f4b169e932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:35.063666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38056a11-ca37-45dc-abe2-42f4b169e932 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:35.077881 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 789/1591] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:49:35.078852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2536cf0-955f-49bd-96c2-f2d61017a265 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:35.089471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2536cf0-955f-49bd-96c2-f2d61017a265 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:35.094624 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5ec46-8dde-45a7-9e2d-89354c988e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:35.095493 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5ec46-8dde-45a7-9e2d-89354c988e46 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:35.095780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5ec46-8dde-45a7-9e2d-89354c988e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:35.096107 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5ec46-8dde-45a7-9e2d-89354c988e46 admin admin] PATH_INFO: `/v3/projects/a5f8ad5cd30d417bad9bebdf6fadbb97/users/0f1cded746ca4d4b93c57260b05eaa8b/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:35.113097 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5ec46-8dde-45a7-9e2d-89354c988e46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5f8ad5cd30d417bad9bebdf6fadbb97, user_id=0f1cded746ca4d4b93c57260b05eaa8b, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:35.115945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5ec46-8dde-45a7-9e2d-89354c988e46 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:35.132113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 803/1592] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:49:35 2022] PUT /identity/v3/projects/a5f8ad5cd30d417bad9bebdf6fadbb97/users/0f1cded746ca4d4b93c57260b05eaa8b/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:49:35.143778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4b8b6-6fe6-40b8-8887-b8aeebc8292f None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:35.144106 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4b8b6-6fe6-40b8-8887-b8aeebc8292f None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:35.144403 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4b8b6-6fe6-40b8-8887-b8aeebc8292f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:35.174495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-4ef4b8b6-6fe6-40b8-8887-b8aeebc8292f None None] MFA Rules not processed for user `0f1cded746ca4d4b93c57260b05eaa8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:35.175721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4ef4b8b6-6fe6-40b8-8887-b8aeebc8292f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:35.246278 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 790/1593] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:49:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:49:35.258703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5ec46-8dde-45a7-9e2d-89354c988e46 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:35.260775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e8d5ec46-8dde-45a7-9e2d-89354c988e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:35.334548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5ec46-8dde-45a7-9e2d-89354c988e46 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:35.338148 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399977ee-2337-4a26-939b-db92ba3dd8e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:35.338760 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399977ee-2337-4a26-939b-db92ba3dd8e5 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:35.339039 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399977ee-2337-4a26-939b-db92ba3dd8e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:35.339308 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399977ee-2337-4a26-939b-db92ba3dd8e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:35.348806 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399977ee-2337-4a26-939b-db92ba3dd8e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:35.350229 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399977ee-2337-4a26-939b-db92ba3dd8e5 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:35.360054 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 804/1594] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:49:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:49:45.243114 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:45.355007 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:55.895610 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4b8b6-6fe6-40b8-8887-b8aeebc8292f None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:55.897103 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4ef4b8b6-6fe6-40b8-8887-b8aeebc8292f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:55.985057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4b8b6-6fe6-40b8-8887-b8aeebc8292f None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:55.990041 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bdb9a-a80c-4625-bd17-b816a0075ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:55.990851 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bdb9a-a80c-4625-bd17-b816a0075ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:55.991136 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bdb9a-a80c-4625-bd17-b816a0075ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:55.991411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bdb9a-a80c-4625-bd17-b816a0075ae5 admin admin] PATH_INFO: `/v3/users/cfe7374737a04042904d4a87469046c8` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:55.992607 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bdb9a-a80c-4625-bd17-b816a0075ae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe7374737a04042904d4a87469046c8)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:55.994945 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bdb9a-a80c-4625-bd17-b816a0075ae5 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:56.068635 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a63bdb9a-a80c-4625-bd17-b816a0075ae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe7374737a04042904d4a87469046c8'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:56.069626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a63bdb9a-a80c-4625-bd17-b816a0075ae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe7374737a04042904d4a87469046c8'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:56.085171 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a63bdb9a-a80c-4625-bd17-b816a0075ae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe7374737a04042904d4a87469046c8'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:56.094632 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a63bdb9a-a80c-4625-bd17-b816a0075ae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe7374737a04042904d4a87469046c8'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:56.095643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a63bdb9a-a80c-4625-bd17-b816a0075ae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe7374737a04042904d4a87469046c8'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:56.112437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 791/1595] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:49:55 2022] DELETE /identity/v3/users/cfe7374737a04042904d4a87469046c8 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:56.117125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9947c-ed6c-47dd-aaae-0c036470cb5e None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:56.117656 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9947c-ed6c-47dd-aaae-0c036470cb5e None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:56.117656 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9947c-ed6c-47dd-aaae-0c036470cb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:56.143381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-92f9947c-ed6c-47dd-aaae-0c036470cb5e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:56.145482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-92f9947c-ed6c-47dd-aaae-0c036470cb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:56.218631 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 805/1596] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:49:56.242449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bdb9a-a80c-4625-bd17-b816a0075ae5 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:56.242449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a63bdb9a-a80c-4625-bd17-b816a0075ae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:56.291838 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bdb9a-a80c-4625-bd17-b816a0075ae5 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:56.295504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987a8a4-fb86-4cca-9c9b-06d642bf9587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:56.296252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987a8a4-fb86-4cca-9c9b-06d642bf9587 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:56.296535 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987a8a4-fb86-4cca-9c9b-06d642bf9587 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:56.296808 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987a8a4-fb86-4cca-9c9b-06d642bf9587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:56.307020 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b987a8a4-fb86-4cca-9c9b-06d642bf9587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:56.308508 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b987a8a4-fb86-4cca-9c9b-06d642bf9587 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:56.318730 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 792/1597] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:49:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:49:56.671352 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9947c-ed6c-47dd-aaae-0c036470cb5e None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:56.674202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-92f9947c-ed6c-47dd-aaae-0c036470cb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:56.686818 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9947c-ed6c-47dd-aaae-0c036470cb5e None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:56.692207 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2facd1-015b-4ea6-8d28-994089bbb085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:56.692872 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2facd1-015b-4ea6-8d28-994089bbb085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:56.693150 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2facd1-015b-4ea6-8d28-994089bbb085 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:56.693422 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2facd1-015b-4ea6-8d28-994089bbb085 admin admin] PATH_INFO: `/v3/projects/19f20f48a5fc45aa829880ed1d7950d0` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:56.694540 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2facd1-015b-4ea6-8d28-994089bbb085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19f20f48a5fc45aa829880ed1d7950d0)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:56.696567 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2facd1-015b-4ea6-8d28-994089bbb085 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:56.755452 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-2a2facd1-015b-4ea6-8d28-994089bbb085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19f20f48a5fc45aa829880ed1d7950d0', 'request_id': 'req-2a2facd1-015b-4ea6-8d28-994089bbb085'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:56.766285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-2a2facd1-015b-4ea6-8d28-994089bbb085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19f20f48a5fc45aa829880ed1d7950d0', 'request_id': 'req-2a2facd1-015b-4ea6-8d28-994089bbb085'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:56.776036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-2a2facd1-015b-4ea6-8d28-994089bbb085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19f20f48a5fc45aa829880ed1d7950d0', 'request_id': 'req-2a2facd1-015b-4ea6-8d28-994089bbb085'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:56.788858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-2a2facd1-015b-4ea6-8d28-994089bbb085 admin admin] The token cache is being invalidate because project 19f20f48a5fc45aa829880ed1d7950d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:49:56.789153 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-2a2facd1-015b-4ea6-8d28-994089bbb085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:56.789825 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-2a2facd1-015b-4ea6-8d28-994089bbb085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:56.791328 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 806/1598] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:49:56 2022] DELETE /identity/v3/projects/19f20f48a5fc45aa829880ed1d7950d0 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:56.795603 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987a8a4-fb86-4cca-9c9b-06d642bf9587 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:56.806252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987a8a4-fb86-4cca-9c9b-06d642bf9587 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:56.835100 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:56.897503 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c78a1-08b3-4ae2-9b1b-4068bca8669e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:56.898172 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c78a1-08b3-4ae2-9b1b-4068bca8669e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:56.898463 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c78a1-08b3-4ae2-9b1b-4068bca8669e admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:56.898740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c78a1-08b3-4ae2-9b1b-4068bca8669e admin admin] PATH_INFO: `/v3/users/b189afa40b9d4e2097f31537cc8f9ae3` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:56.899900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c78a1-08b3-4ae2-9b1b-4068bca8669e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b189afa40b9d4e2097f31537cc8f9ae3)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:56.901964 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c78a1-08b3-4ae2-9b1b-4068bca8669e admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:56.962710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-c15c78a1-08b3-4ae2-9b1b-4068bca8669e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b189afa40b9d4e2097f31537cc8f9ae3'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:56.963446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-c15c78a1-08b3-4ae2-9b1b-4068bca8669e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b189afa40b9d4e2097f31537cc8f9ae3'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:56.976979 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-c15c78a1-08b3-4ae2-9b1b-4068bca8669e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b189afa40b9d4e2097f31537cc8f9ae3'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:56.989345 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-c15c78a1-08b3-4ae2-9b1b-4068bca8669e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b189afa40b9d4e2097f31537cc8f9ae3'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:56.990420 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-c15c78a1-08b3-4ae2-9b1b-4068bca8669e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b189afa40b9d4e2097f31537cc8f9ae3'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:57.014611 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 793/1599] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:49:56 2022] DELETE /identity/v3/users/b189afa40b9d4e2097f31537cc8f9ae3 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:57.183532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2facd1-015b-4ea6-8d28-994089bbb085 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:57.185279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-2a2facd1-015b-4ea6-8d28-994089bbb085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:57.200740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2facd1-015b-4ea6-8d28-994089bbb085 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:57.339193 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2616a-b076-46b3-b1ae-cfbb3f435a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:57.340124 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2616a-b076-46b3-b1ae-cfbb3f435a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:57.341392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2616a-b076-46b3-b1ae-cfbb3f435a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:57.342039 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2616a-b076-46b3-b1ae-cfbb3f435a39 admin admin] PATH_INFO: `/v3/users/41613d6504334fd6aca800fe9ee6526b` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:57.346471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2616a-b076-46b3-b1ae-cfbb3f435a39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41613d6504334fd6aca800fe9ee6526b)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:57.349702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2616a-b076-46b3-b1ae-cfbb3f435a39 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:57.412553 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:57.430748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c78a1-08b3-4ae2-9b1b-4068bca8669e admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:57.432205 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c15c78a1-08b3-4ae2-9b1b-4068bca8669e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:57.455022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-1aa2616a-b076-46b3-b1ae-cfbb3f435a39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41613d6504334fd6aca800fe9ee6526b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:57.455799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-1aa2616a-b076-46b3-b1ae-cfbb3f435a39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41613d6504334fd6aca800fe9ee6526b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:57.472376 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c78a1-08b3-4ae2-9b1b-4068bca8669e admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:57.482515 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0106b0-cb40-4ffe-a7df-23b01e84b449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:57.483805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0106b0-cb40-4ffe-a7df-23b01e84b449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:57.483928 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-1aa2616a-b076-46b3-b1ae-cfbb3f435a39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41613d6504334fd6aca800fe9ee6526b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:57.484149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0106b0-cb40-4ffe-a7df-23b01e84b449 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:57.484408 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0106b0-cb40-4ffe-a7df-23b01e84b449 admin admin] PATH_INFO: `/v3/projects/e42199fcb9a34be7a2bb49bd5c6d27e5` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:57.485636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0106b0-cb40-4ffe-a7df-23b01e84b449 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e42199fcb9a34be7a2bb49bd5c6d27e5)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:57.487926 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0106b0-cb40-4ffe-a7df-23b01e84b449 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:57.494428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-1aa2616a-b076-46b3-b1ae-cfbb3f435a39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41613d6504334fd6aca800fe9ee6526b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:57.504949 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-1aa2616a-b076-46b3-b1ae-cfbb3f435a39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41613d6504334fd6aca800fe9ee6526b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:57.517224 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 807/1600] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:49:57 2022] DELETE /identity/v3/users/41613d6504334fd6aca800fe9ee6526b => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:57.520426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f95e2d-f553-4de5-91d8-f91496a3a87b None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:57.520569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f95e2d-f553-4de5-91d8-f91496a3a87b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:57.520907 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f95e2d-f553-4de5-91d8-f91496a3a87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:57.545119 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-59f95e2d-f553-4de5-91d8-f91496a3a87b None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:57.547075 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-59f95e2d-f553-4de5-91d8-f91496a3a87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:57.557305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6d0106b0-cb40-4ffe-a7df-23b01e84b449 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e42199fcb9a34be7a2bb49bd5c6d27e5', 'request_id': 'req-6d0106b0-cb40-4ffe-a7df-23b01e84b449'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:57.570945 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6d0106b0-cb40-4ffe-a7df-23b01e84b449 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e42199fcb9a34be7a2bb49bd5c6d27e5', 'request_id': 'req-6d0106b0-cb40-4ffe-a7df-23b01e84b449'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:57.579221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6d0106b0-cb40-4ffe-a7df-23b01e84b449 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e42199fcb9a34be7a2bb49bd5c6d27e5', 'request_id': 'req-6d0106b0-cb40-4ffe-a7df-23b01e84b449'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:57.593687 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6d0106b0-cb40-4ffe-a7df-23b01e84b449 admin admin] The token cache is being invalidate because project e42199fcb9a34be7a2bb49bd5c6d27e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:49:57.593989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6d0106b0-cb40-4ffe-a7df-23b01e84b449 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:57.594681 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6d0106b0-cb40-4ffe-a7df-23b01e84b449 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:57.596473 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 794/1601] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:49:57 2022] DELETE /identity/v3/projects/e42199fcb9a34be7a2bb49bd5c6d27e5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:57.602220 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c092f67-e3ae-40ed-92b3-103fca844f32 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:57.602506 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c092f67-e3ae-40ed-92b3-103fca844f32 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:57.602856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c092f67-e3ae-40ed-92b3-103fca844f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:57.626045 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-3c092f67-e3ae-40ed-92b3-103fca844f32 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:57.627332 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-3c092f67-e3ae-40ed-92b3-103fca844f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:57.635815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 808/1602] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:49:57.637818 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 795/1603] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:49:57.646487 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd33196-1134-4898-9ebf-e3a7b10c39f3 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:57.646852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd33196-1134-4898-9ebf-e3a7b10c39f3 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:57.647148 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd33196-1134-4898-9ebf-e3a7b10c39f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:57.648595 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c092f67-e3ae-40ed-92b3-103fca844f32 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:57.650551 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-3c092f67-e3ae-40ed-92b3-103fca844f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:57.677534 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:57.678857 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-7fd33196-1134-4898-9ebf-e3a7b10c39f3 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:57.680201 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-7fd33196-1134-4898-9ebf-e3a7b10c39f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:57.697340 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 809/1604] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:49:57.698675 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd33196-1134-4898-9ebf-e3a7b10c39f3 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:57.729669 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd33196-1134-4898-9ebf-e3a7b10c39f3 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:57.736272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b058a-dde8-4f82-be1e-6fbe2d10f263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:57.737106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b058a-dde8-4f82-be1e-6fbe2d10f263 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:57.737346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b058a-dde8-4f82-be1e-6fbe2d10f263 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:57.737623 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b058a-dde8-4f82-be1e-6fbe2d10f263 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:57.738626 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b058a-dde8-4f82-be1e-6fbe2d10f263 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:57.739109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b058a-dde8-4f82-be1e-6fbe2d10f263 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:57.741392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b058a-dde8-4f82-be1e-6fbe2d10f263 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:57.744389 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c092f67-e3ae-40ed-92b3-103fca844f32 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:57.750541 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eec9d7-70c8-4a04-82a9-867905ee5e5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:57.751522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eec9d7-70c8-4a04-82a9-867905ee5e5a service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:57.751977 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eec9d7-70c8-4a04-82a9-867905ee5e5a service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:57.752495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eec9d7-70c8-4a04-82a9-867905ee5e5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:57.759354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 810/1605] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:49:57.769642 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eec9d7-70c8-4a04-82a9-867905ee5e5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:57.771759 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eec9d7-70c8-4a04-82a9-867905ee5e5a service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:57.777976 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b058a-dde8-4f82-be1e-6fbe2d10f263 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:57.790681 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 796/1606] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:49:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:49:57.802392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b058a-dde8-4f82-be1e-6fbe2d10f263 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:57.822699 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48deb6a-9f53-4fb2-84df-70a6ec65ded6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:57.823492 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48deb6a-9f53-4fb2-84df-70a6ec65ded6 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:57.823854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48deb6a-9f53-4fb2-84df-70a6ec65ded6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:57.824330 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48deb6a-9f53-4fb2-84df-70a6ec65ded6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:57.825236 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48deb6a-9f53-4fb2-84df-70a6ec65ded6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:57.827283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48deb6a-9f53-4fb2-84df-70a6ec65ded6 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:57.885626 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 811/1607] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:49:57 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:49:57.901490 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eec9d7-70c8-4a04-82a9-867905ee5e5a service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:57.917835 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eec9d7-70c8-4a04-82a9-867905ee5e5a service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:58.075492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038f15a-2601-454b-84ba-bda8c8816264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:58.076236 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038f15a-2601-454b-84ba-bda8c8816264 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.076744 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038f15a-2601-454b-84ba-bda8c8816264 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.077254 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038f15a-2601-454b-84ba-bda8c8816264 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.078006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3038f15a-2601-454b-84ba-bda8c8816264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:58.080048 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3038f15a-2601-454b-84ba-bda8c8816264 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:58.124712 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 797/1608] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:49:57 2022] POST /identity/v3/users => generated 294 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:49:58.130720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48deb6a-9f53-4fb2-84df-70a6ec65ded6 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:58.142847 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48deb6a-9f53-4fb2-84df-70a6ec65ded6 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:58.148062 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b7208-1838-4a7c-977a-9d5b537eccd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:58.148751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b7208-1838-4a7c-977a-9d5b537eccd0 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.149029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b7208-1838-4a7c-977a-9d5b537eccd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.149372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b7208-1838-4a7c-977a-9d5b537eccd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.150434 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b7208-1838-4a7c-977a-9d5b537eccd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:58.152070 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b7208-1838-4a7c-977a-9d5b537eccd0 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:58.173724 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038f15a-2601-454b-84ba-bda8c8816264 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:58.175007 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 812/1609] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:49:58.205890 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038f15a-2601-454b-84ba-bda8c8816264 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:58.212009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b5559-ba07-4af7-993e-85dc626786c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:58.213018 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b5559-ba07-4af7-993e-85dc626786c8 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.213990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b5559-ba07-4af7-993e-85dc626786c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.214848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b5559-ba07-4af7-993e-85dc626786c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.215713 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b5559-ba07-4af7-993e-85dc626786c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:58.219541 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b5559-ba07-4af7-993e-85dc626786c8 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:58.240839 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b7208-1838-4a7c-977a-9d5b537eccd0 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:58.249736 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 798/1610] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:49:58.253879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b7208-1838-4a7c-977a-9d5b537eccd0 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:58.262189 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a5086-19c5-4370-a854-cdba789bc9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:58.262992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a5086-19c5-4370-a854-cdba789bc9b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.263454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a5086-19c5-4370-a854-cdba789bc9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.263950 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a5086-19c5-4370-a854-cdba789bc9b9 admin admin] PATH_INFO: `/v3/projects/da7d1bae8f31427cbbb6405f13e58b5a/users/ff7af7e7c32e45aea44a723739132175/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.299761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a5086-19c5-4370-a854-cdba789bc9b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da7d1bae8f31427cbbb6405f13e58b5a, user_id=ff7af7e7c32e45aea44a723739132175, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:58.302655 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a5086-19c5-4370-a854-cdba789bc9b9 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:58.319780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 813/1611] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:49:58 2022] PUT /identity/v3/projects/da7d1bae8f31427cbbb6405f13e58b5a/users/ff7af7e7c32e45aea44a723739132175/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:49:58.332149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb198b12-0b54-431c-87fe-024bbe5c7b27 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.332442 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb198b12-0b54-431c-87fe-024bbe5c7b27 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.332792 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb198b12-0b54-431c-87fe-024bbe5c7b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.373427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-bb198b12-0b54-431c-87fe-024bbe5c7b27 None None] MFA Rules not processed for user `ff7af7e7c32e45aea44a723739132175`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:58.375772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-bb198b12-0b54-431c-87fe-024bbe5c7b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:58.461762 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 799/1612] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:49:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:49:58.466890 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a5086-19c5-4370-a854-cdba789bc9b9 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:58.503265 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:58.515585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb198b12-0b54-431c-87fe-024bbe5c7b27 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:58.517300 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-bb198b12-0b54-431c-87fe-024bbe5c7b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:58.555167 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:58.556989 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a5086-19c5-4370-a854-cdba789bc9b9 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:58.564763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0783c3-fade-445c-9b36-b0e1642d4858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:58.565387 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0783c3-fade-445c-9b36-b0e1642d4858 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.565675 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0783c3-fade-445c-9b36-b0e1642d4858 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.566093 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0783c3-fade-445c-9b36-b0e1642d4858 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.566745 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0783c3-fade-445c-9b36-b0e1642d4858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:58.593765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0783c3-fade-445c-9b36-b0e1642d4858 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:58.621314 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb198b12-0b54-431c-87fe-024bbe5c7b27 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:58.631359 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbe1df-a095-4eb2-96d3-2b2c7af77a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:58.632060 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 814/1613] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:49:58 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:49:58.632631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbe1df-a095-4eb2-96d3-2b2c7af77a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.633055 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbe1df-a095-4eb2-96d3-2b2c7af77a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.633601 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbe1df-a095-4eb2-96d3-2b2c7af77a05 admin admin] PATH_INFO: `/v3/projects/10ae58bd73364dc0965b6ab60119ef53` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.635629 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbe1df-a095-4eb2-96d3-2b2c7af77a05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10ae58bd73364dc0965b6ab60119ef53)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:58.636064 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0783c3-fade-445c-9b36-b0e1642d4858 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:58.640373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbe1df-a095-4eb2-96d3-2b2c7af77a05 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:58.745373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-badbe1df-a095-4eb2-96d3-2b2c7af77a05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10ae58bd73364dc0965b6ab60119ef53', 'request_id': 'req-badbe1df-a095-4eb2-96d3-2b2c7af77a05'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:58.761300 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-badbe1df-a095-4eb2-96d3-2b2c7af77a05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10ae58bd73364dc0965b6ab60119ef53', 'request_id': 'req-badbe1df-a095-4eb2-96d3-2b2c7af77a05'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:58.764138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0783c3-fade-445c-9b36-b0e1642d4858 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:58.771264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-badbe1df-a095-4eb2-96d3-2b2c7af77a05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10ae58bd73364dc0965b6ab60119ef53', 'request_id': 'req-badbe1df-a095-4eb2-96d3-2b2c7af77a05'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:58.784329 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-badbe1df-a095-4eb2-96d3-2b2c7af77a05 admin admin] The token cache is being invalidate because project 10ae58bd73364dc0965b6ab60119ef53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:49:58.784648 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-badbe1df-a095-4eb2-96d3-2b2c7af77a05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:58.785651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-badbe1df-a095-4eb2-96d3-2b2c7af77a05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:58.789975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 800/1614] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:49:58 2022] DELETE /identity/v3/projects/10ae58bd73364dc0965b6ab60119ef53 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:49:58.790951 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbe1df-a095-4eb2-96d3-2b2c7af77a05 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:58.792282 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-badbe1df-a095-4eb2-96d3-2b2c7af77a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:58.836349 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e32a8-f9a4-4ecf-bdcf-63aef626cdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:58.836969 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e32a8-f9a4-4ecf-bdcf-63aef626cdf5 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.837895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e32a8-f9a4-4ecf-bdcf-63aef626cdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.838179 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e32a8-f9a4-4ecf-bdcf-63aef626cdf5 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.838815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e32a8-f9a4-4ecf-bdcf-63aef626cdf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:58.840813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e32a8-f9a4-4ecf-bdcf-63aef626cdf5 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:58.862851 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 815/1615] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:49:58 2022] POST /identity/v3/users => generated 300 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:49:58.864528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbe1df-a095-4eb2-96d3-2b2c7af77a05 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:58.866495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e32a8-f9a4-4ecf-bdcf-63aef626cdf5 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:58.868165 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-026e32a8-f9a4-4ecf-bdcf-63aef626cdf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:58.876311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc039e-c61d-4190-aa9b-f4a89e4266b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:58.876965 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc039e-c61d-4190-aa9b-f4a89e4266b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.877644 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc039e-c61d-4190-aa9b-f4a89e4266b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.877922 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc039e-c61d-4190-aa9b-f4a89e4266b0 admin admin] PATH_INFO: `/v3/users/ad494471d3304ff393efe693b0b52524` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.879077 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc039e-c61d-4190-aa9b-f4a89e4266b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad494471d3304ff393efe693b0b52524)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:58.880683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e32a8-f9a4-4ecf-bdcf-63aef626cdf5 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:58.881723 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc039e-c61d-4190-aa9b-f4a89e4266b0 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:58.885480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc969c1-8fef-4e8f-bcbe-ef94bdea4889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:58.886181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc969c1-8fef-4e8f-bcbe-ef94bdea4889 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.886535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc969c1-8fef-4e8f-bcbe-ef94bdea4889 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.886954 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc969c1-8fef-4e8f-bcbe-ef94bdea4889 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.887694 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc969c1-8fef-4e8f-bcbe-ef94bdea4889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:58.890133 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc969c1-8fef-4e8f-bcbe-ef94bdea4889 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:58.903256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 816/1616] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:49:58.909271 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc969c1-8fef-4e8f-bcbe-ef94bdea4889 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:58.922421 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc969c1-8fef-4e8f-bcbe-ef94bdea4889 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:58.927130 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701361e2-9241-4686-ac90-c3cf6795e5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:58.927795 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701361e2-9241-4686-ac90-c3cf6795e5ad admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:58.928101 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701361e2-9241-4686-ac90-c3cf6795e5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:58.928381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701361e2-9241-4686-ac90-c3cf6795e5ad admin admin] PATH_INFO: `/v3/projects/f04b45a5dfa843aab553b3ac4306387b/users/7cf2b2a3bc324b2e9c436efcf92d9726/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:58.937892 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-bbdc039e-c61d-4190-aa9b-f4a89e4266b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad494471d3304ff393efe693b0b52524'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:58.939537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-bbdc039e-c61d-4190-aa9b-f4a89e4266b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad494471d3304ff393efe693b0b52524'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:58.945341 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701361e2-9241-4686-ac90-c3cf6795e5ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=f04b45a5dfa843aab553b3ac4306387b, user_id=7cf2b2a3bc324b2e9c436efcf92d9726, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:58.948104 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701361e2-9241-4686-ac90-c3cf6795e5ad admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:58.950973 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:58.953804 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-bbdc039e-c61d-4190-aa9b-f4a89e4266b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad494471d3304ff393efe693b0b52524'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:58.963600 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-bbdc039e-c61d-4190-aa9b-f4a89e4266b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad494471d3304ff393efe693b0b52524'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:58.965609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-bbdc039e-c61d-4190-aa9b-f4a89e4266b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad494471d3304ff393efe693b0b52524'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:58.971796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 817/1617] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:49:58 2022] PUT /identity/v3/projects/f04b45a5dfa843aab553b3ac4306387b/users/7cf2b2a3bc324b2e9c436efcf92d9726/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:58.975727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701361e2-9241-4686-ac90-c3cf6795e5ad admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:58.977354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-701361e2-9241-4686-ac90-c3cf6795e5ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:58.978793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 801/1618] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:49:58 2022] DELETE /identity/v3/users/ad494471d3304ff393efe693b0b52524 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:49:58.997027 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:59.059276 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701361e2-9241-4686-ac90-c3cf6795e5ad admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.120350 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa20a7-939b-4043-9738-ccc288445116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.120977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa20a7-939b-4043-9738-ccc288445116 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.121266 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa20a7-939b-4043-9738-ccc288445116 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.121542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa20a7-939b-4043-9738-ccc288445116 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.122186 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa20a7-939b-4043-9738-ccc288445116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:59.124409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa20a7-939b-4043-9738-ccc288445116 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:59.134445 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 818/1619] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:49:59.138613 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc039e-c61d-4190-aa9b-f4a89e4266b0 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.151524 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc039e-c61d-4190-aa9b-f4a89e4266b0 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.156898 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3e0db-66c1-4679-992d-1033be4d20b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.157811 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3e0db-66c1-4679-992d-1033be4d20b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.158161 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3e0db-66c1-4679-992d-1033be4d20b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.158526 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3e0db-66c1-4679-992d-1033be4d20b6 admin admin] PATH_INFO: `/v3/projects/f04b45a5dfa843aab553b3ac4306387b/users/7cf2b2a3bc324b2e9c436efcf92d9726/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.160593 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3e0db-66c1-4679-992d-1033be4d20b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f04b45a5dfa843aab553b3ac4306387b, user_id=7cf2b2a3bc324b2e9c436efcf92d9726, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:59.163269 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3e0db-66c1-4679-992d-1033be4d20b6 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:59.174673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 802/1620] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:49:59 2022] PUT /identity/v3/projects/f04b45a5dfa843aab553b3ac4306387b/users/7cf2b2a3bc324b2e9c436efcf92d9726/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:59.184825 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c836c1c-0df1-471f-83ea-1e13937af10b None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.185139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c836c1c-0df1-471f-83ea-1e13937af10b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.185442 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c836c1c-0df1-471f-83ea-1e13937af10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.214813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-7c836c1c-0df1-471f-83ea-1e13937af10b None None] MFA Rules not processed for user `7cf2b2a3bc324b2e9c436efcf92d9726`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:59.216043 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-7c836c1c-0df1-471f-83ea-1e13937af10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:59.282659 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3e0db-66c1-4679-992d-1033be4d20b6 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.284696 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-2dd3e0db-66c1-4679-992d-1033be4d20b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:59.293965 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 819/1621] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:49:59 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:49:59.315514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c836c1c-0df1-471f-83ea-1e13937af10b None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.317041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-7c836c1c-0df1-471f-83ea-1e13937af10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:59.372262 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:59.390893 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c836c1c-0df1-471f-83ea-1e13937af10b None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.396616 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c07b0-f623-4807-9a5a-9dbefb69b9e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.397243 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c07b0-f623-4807-9a5a-9dbefb69b9e2 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.397587 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c07b0-f623-4807-9a5a-9dbefb69b9e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.397998 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c07b0-f623-4807-9a5a-9dbefb69b9e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.408370 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c07b0-f623-4807-9a5a-9dbefb69b9e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:59.409956 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c07b0-f623-4807-9a5a-9dbefb69b9e2 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:59.412732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3e0db-66c1-4679-992d-1033be4d20b6 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.418057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4bd09-5033-4acd-b985-7ee929ece44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.418690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4bd09-5033-4acd-b985-7ee929ece44f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.418966 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4bd09-5033-4acd-b985-7ee929ece44f admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.419233 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4bd09-5033-4acd-b985-7ee929ece44f admin admin] PATH_INFO: `/v3/projects/82297cb747944853a03350f2aed55bb1` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.420578 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4bd09-5033-4acd-b985-7ee929ece44f admin admin] RBAC: Authorizing `identity:delete_project(project_id=82297cb747944853a03350f2aed55bb1)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:59.423603 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4bd09-5033-4acd-b985-7ee929ece44f admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:59.428324 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 820/1622] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:49:59 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:49:59.496239 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c07b0-f623-4807-9a5a-9dbefb69b9e2 service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.502245 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-c7c4bd09-5033-4acd-b985-7ee929ece44f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82297cb747944853a03350f2aed55bb1', 'request_id': 'req-c7c4bd09-5033-4acd-b985-7ee929ece44f'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:59.511494 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c07b0-f623-4807-9a5a-9dbefb69b9e2 service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.511679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-c7c4bd09-5033-4acd-b985-7ee929ece44f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82297cb747944853a03350f2aed55bb1', 'request_id': 'req-c7c4bd09-5033-4acd-b985-7ee929ece44f'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:59.520549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-c7c4bd09-5033-4acd-b985-7ee929ece44f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82297cb747944853a03350f2aed55bb1', 'request_id': 'req-c7c4bd09-5033-4acd-b985-7ee929ece44f'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:59.543188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-c7c4bd09-5033-4acd-b985-7ee929ece44f admin admin] The token cache is being invalidate because project 82297cb747944853a03350f2aed55bb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:49:59.543615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-c7c4bd09-5033-4acd-b985-7ee929ece44f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:59.544395 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-c7c4bd09-5033-4acd-b985-7ee929ece44f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:59.546848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 803/1623] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:49:59 2022] DELETE /identity/v3/projects/82297cb747944853a03350f2aed55bb1 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:49:59.552016 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1472b4-919f-4034-a539-dcc6201a9b29 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.552345 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1472b4-919f-4034-a539-dcc6201a9b29 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.552623 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1472b4-919f-4034-a539-dcc6201a9b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.579437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-2b1472b4-919f-4034-a539-dcc6201a9b29 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:59.581096 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-2b1472b4-919f-4034-a539-dcc6201a9b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:59.586207 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd8112-7772-46de-b72a-4a318f7456a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.587060 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd8112-7772-46de-b72a-4a318f7456a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.587339 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd8112-7772-46de-b72a-4a318f7456a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.587611 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd8112-7772-46de-b72a-4a318f7456a2 admin admin] PATH_INFO: `/v3/users/ff7af7e7c32e45aea44a723739132175` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.588765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd8112-7772-46de-b72a-4a318f7456a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff7af7e7c32e45aea44a723739132175)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:59.590838 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd8112-7772-46de-b72a-4a318f7456a2 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:59.596328 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 804/1624] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:49:59.600151 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f6d1e-6ec0-4fb8-b552-1c198e78b1ef None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.600746 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f6d1e-6ec0-4fb8-b552-1c198e78b1ef None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.601090 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f6d1e-6ec0-4fb8-b552-1c198e78b1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.626948 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-fd0f6d1e-6ec0-4fb8-b552-1c198e78b1ef None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:59.628521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-fd0f6d1e-6ec0-4fb8-b552-1c198e78b1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:59.640220 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 805/1625] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:49:59.643580 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f6d1e-6ec0-4fb8-b552-1c198e78b1ef None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.645550 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-1fdd8112-7772-46de-b72a-4a318f7456a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff7af7e7c32e45aea44a723739132175'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:59.646210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-1fdd8112-7772-46de-b72a-4a318f7456a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff7af7e7c32e45aea44a723739132175'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:59.653587 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f6d1e-6ec0-4fb8-b552-1c198e78b1ef None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.658912 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c6ea8-1371-48d4-9a3e-0ab463925775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.659058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-1fdd8112-7772-46de-b72a-4a318f7456a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff7af7e7c32e45aea44a723739132175'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:59.659552 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c6ea8-1371-48d4-9a3e-0ab463925775 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.659830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c6ea8-1371-48d4-9a3e-0ab463925775 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.660112 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c6ea8-1371-48d4-9a3e-0ab463925775 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.660776 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c6ea8-1371-48d4-9a3e-0ab463925775 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:49:59.661073 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c6ea8-1371-48d4-9a3e-0ab463925775 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:59.662647 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c6ea8-1371-48d4-9a3e-0ab463925775 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:59.666197 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-1fdd8112-7772-46de-b72a-4a318f7456a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff7af7e7c32e45aea44a723739132175'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:59.675086 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-1fdd8112-7772-46de-b72a-4a318f7456a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff7af7e7c32e45aea44a723739132175'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:49:59.675253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 806/1626] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:49:59.679441 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c6ea8-1371-48d4-9a3e-0ab463925775 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.681179 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-3a1c6ea8-1371-48d4-9a3e-0ab463925775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:59.682256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 821/1627] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:49:59 2022] DELETE /identity/v3/users/ff7af7e7c32e45aea44a723739132175 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:49:59.686517 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caf83a-e62e-41d5-9c03-9cc4bc2054b6 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.686810 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caf83a-e62e-41d5-9c03-9cc4bc2054b6 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.687073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caf83a-e62e-41d5-9c03-9cc4bc2054b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.695370 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c6ea8-1371-48d4-9a3e-0ab463925775 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.706571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-94caf83a-e62e-41d5-9c03-9cc4bc2054b6 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:49:59.707764 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-94caf83a-e62e-41d5-9c03-9cc4bc2054b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:59.726837 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:59.761411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5239bf1-5e6e-489d-948c-353011e82cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.762030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5239bf1-5e6e-489d-948c-353011e82cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.762312 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5239bf1-5e6e-489d-948c-353011e82cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.762582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5239bf1-5e6e-489d-948c-353011e82cf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.763212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5239bf1-5e6e-489d-948c-353011e82cf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:59.765968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5239bf1-5e6e-489d-948c-353011e82cf5 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:59.786243 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 822/1628] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:49:59.793787 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 807/1629] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:49:59 2022] POST /identity/v3/projects => generated 357 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:49:59.797987 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5239bf1-5e6e-489d-948c-353011e82cf5 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.798173 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caf83a-e62e-41d5-9c03-9cc4bc2054b6 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.799618 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-94caf83a-e62e-41d5-9c03-9cc4bc2054b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:49:59.805934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:49:59.860550 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caf83a-e62e-41d5-9c03-9cc4bc2054b6 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.863035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5239bf1-5e6e-489d-948c-353011e82cf5 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.867668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84bed2-ab69-4b86-a23d-9880f64000ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.868306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84bed2-ab69-4b86-a23d-9880f64000ed admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.868631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84bed2-ab69-4b86-a23d-9880f64000ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.872541 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84bed2-ab69-4b86-a23d-9880f64000ed admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.873669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84bed2-ab69-4b86-a23d-9880f64000ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:59.876395 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84bed2-ab69-4b86-a23d-9880f64000ed admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:59.901914 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 808/1630] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:49:59 2022] POST /identity/v3/users => generated 279 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:49:59.906755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84bed2-ab69-4b86-a23d-9880f64000ed admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.914320 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168680b-298d-4b9b-b066-ff70b18ca2cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.914931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168680b-298d-4b9b-b066-ff70b18ca2cb service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.915211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168680b-298d-4b9b-b066-ff70b18ca2cb service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.915486 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168680b-298d-4b9b-b066-ff70b18ca2cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.916304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84bed2-ab69-4b86-a23d-9880f64000ed admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.920912 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7dac5-9726-4236-aa50-20b7759233cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.921503 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7dac5-9726-4236-aa50-20b7759233cb admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.921780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7dac5-9726-4236-aa50-20b7759233cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.922050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7dac5-9726-4236-aa50-20b7759233cb admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.922683 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7dac5-9726-4236-aa50-20b7759233cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:59.924312 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7dac5-9726-4236-aa50-20b7759233cb admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:59.924690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168680b-298d-4b9b-b066-ff70b18ca2cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:59.926103 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168680b-298d-4b9b-b066-ff70b18ca2cb service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:59.934871 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 809/1631] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:49:59.935652 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 823/1632] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:49:59.938744 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7dac5-9726-4236-aa50-20b7759233cb admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.948708 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7dac5-9726-4236-aa50-20b7759233cb admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.954562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a0dc7-9839-46e5-99c4-8bee13c289ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.955297 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a0dc7-9839-46e5-99c4-8bee13c289ac admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.955580 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a0dc7-9839-46e5-99c4-8bee13c289ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.955856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a0dc7-9839-46e5-99c4-8bee13c289ac admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:49:59.956526 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a0dc7-9839-46e5-99c4-8bee13c289ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:49:59.958135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a0dc7-9839-46e5-99c4-8bee13c289ac admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:49:59.969575 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 810/1633] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:49:59.973911 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168680b-298d-4b9b-b066-ff70b18ca2cb service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:49:59.986580 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168680b-298d-4b9b-b066-ff70b18ca2cb service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:49:59.994151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7283cee-4a23-4a94-a4d4-8aae3a67a0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:49:59.995284 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7283cee-4a23-4a94-a4d4-8aae3a67a0f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:49:59.995652 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7283cee-4a23-4a94-a4d4-8aae3a67a0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:49:59.996466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7283cee-4a23-4a94-a4d4-8aae3a67a0f1 admin admin] PATH_INFO: `/v3/projects/35a58dbd858c4153b732ae1b5edeb8de/users/269a69f4f0694ed5b27a1fad9d222691/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:00.018301 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7283cee-4a23-4a94-a4d4-8aae3a67a0f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35a58dbd858c4153b732ae1b5edeb8de, user_id=269a69f4f0694ed5b27a1fad9d222691, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:00.021825 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7283cee-4a23-4a94-a4d4-8aae3a67a0f1 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:00.040559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 824/1634] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:49:59 2022] PUT /identity/v3/projects/35a58dbd858c4153b732ae1b5edeb8de/users/269a69f4f0694ed5b27a1fad9d222691/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:00.052748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc5e48-677a-4cf7-9a50-99f8e2486d70 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:00.053033 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc5e48-677a-4cf7-9a50-99f8e2486d70 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:00.053302 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc5e48-677a-4cf7-9a50-99f8e2486d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:00.101424 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-70cc5e48-677a-4cf7-9a50-99f8e2486d70 None None] MFA Rules not processed for user `269a69f4f0694ed5b27a1fad9d222691`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:00.102957 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-70cc5e48-677a-4cf7-9a50-99f8e2486d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:00.170376 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 811/1635] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:00.174938 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7283cee-4a23-4a94-a4d4-8aae3a67a0f1 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:00.223892 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc5e48-677a-4cf7-9a50-99f8e2486d70 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:00.225306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-70cc5e48-677a-4cf7-9a50-99f8e2486d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:00.239522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:00.247481 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7283cee-4a23-4a94-a4d4-8aae3a67a0f1 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:00.254062 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538fda7-f0cc-43aa-98c7-b2d92f8dcda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:00.254787 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538fda7-f0cc-43aa-98c7-b2d92f8dcda9 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:00.254972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538fda7-f0cc-43aa-98c7-b2d92f8dcda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:00.255237 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538fda7-f0cc-43aa-98c7-b2d92f8dcda9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:00.255872 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538fda7-f0cc-43aa-98c7-b2d92f8dcda9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:00.258265 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538fda7-f0cc-43aa-98c7-b2d92f8dcda9 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:00.287616 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 825/1636] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:00 2022] POST /identity/v3/projects => generated 357 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:00.292130 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538fda7-f0cc-43aa-98c7-b2d92f8dcda9 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:00.298829 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc5e48-677a-4cf7-9a50-99f8e2486d70 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:00.319605 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:00.357165 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538fda7-f0cc-43aa-98c7-b2d92f8dcda9 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:00.361066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ac3e0-f7e6-42df-b7fc-34b78ba540ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:00.361807 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ac3e0-f7e6-42df-b7fc-34b78ba540ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:00.362154 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ac3e0-f7e6-42df-b7fc-34b78ba540ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:00.362487 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff434a34-ea7a-4ed4-9be8-840cd499634d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:00.362610 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ac3e0-f7e6-42df-b7fc-34b78ba540ec admin admin] PATH_INFO: `/v3/projects/da7d1bae8f31427cbbb6405f13e58b5a` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:00.363097 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff434a34-ea7a-4ed4-9be8-840cd499634d admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:00.363378 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff434a34-ea7a-4ed4-9be8-840cd499634d admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:00.363599 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ac3e0-f7e6-42df-b7fc-34b78ba540ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=da7d1bae8f31427cbbb6405f13e58b5a)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:00.363699 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff434a34-ea7a-4ed4-9be8-840cd499634d admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:00.364300 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff434a34-ea7a-4ed4-9be8-840cd499634d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:00.365573 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ac3e0-f7e6-42df-b7fc-34b78ba540ec admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:00.366214 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff434a34-ea7a-4ed4-9be8-840cd499634d admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:00.391884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 826/1637] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:00 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:50:00.396034 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff434a34-ea7a-4ed4-9be8-840cd499634d admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:00.407996 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff434a34-ea7a-4ed4-9be8-840cd499634d admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:00.420241 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e13ac3e0-f7e6-42df-b7fc-34b78ba540ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da7d1bae8f31427cbbb6405f13e58b5a', 'request_id': 'req-e13ac3e0-f7e6-42df-b7fc-34b78ba540ec'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:00.438786 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e13ac3e0-f7e6-42df-b7fc-34b78ba540ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da7d1bae8f31427cbbb6405f13e58b5a', 'request_id': 'req-e13ac3e0-f7e6-42df-b7fc-34b78ba540ec'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:00.445770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e13ac3e0-f7e6-42df-b7fc-34b78ba540ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da7d1bae8f31427cbbb6405f13e58b5a', 'request_id': 'req-e13ac3e0-f7e6-42df-b7fc-34b78ba540ec'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:00.457197 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e13ac3e0-f7e6-42df-b7fc-34b78ba540ec admin admin] The token cache is being invalidate because project da7d1bae8f31427cbbb6405f13e58b5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:00.457489 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e13ac3e0-f7e6-42df-b7fc-34b78ba540ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:00.458150 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e13ac3e0-f7e6-42df-b7fc-34b78ba540ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:00.459893 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 812/1638] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:00 2022] DELETE /identity/v3/projects/da7d1bae8f31427cbbb6405f13e58b5a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:00.464170 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ac3e0-f7e6-42df-b7fc-34b78ba540ec admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:00.465613 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e13ac3e0-f7e6-42df-b7fc-34b78ba540ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:00.467534 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b352467-b4ca-430a-9306-6ba52465f8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:00.468174 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b352467-b4ca-430a-9306-6ba52465f8ab admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:00.468454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b352467-b4ca-430a-9306-6ba52465f8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:00.468726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b352467-b4ca-430a-9306-6ba52465f8ab admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:00.469366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b352467-b4ca-430a-9306-6ba52465f8ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:00.471340 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b352467-b4ca-430a-9306-6ba52465f8ab admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:00.479471 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ac3e0-f7e6-42df-b7fc-34b78ba540ec admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:00.481358 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 827/1639] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:00.485145 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b8124-6c6f-4525-b76a-64e991f6bd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:00.485776 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b8124-6c6f-4525-b76a-64e991f6bd42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:00.486055 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b8124-6c6f-4525-b76a-64e991f6bd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:00.486148 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b352467-b4ca-430a-9306-6ba52465f8ab admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:00.486325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b8124-6c6f-4525-b76a-64e991f6bd42 admin admin] PATH_INFO: `/v3/users/7cf2b2a3bc324b2e9c436efcf92d9726` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:00.487470 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b8124-6c6f-4525-b76a-64e991f6bd42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cf2b2a3bc324b2e9c436efcf92d9726)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:00.487773 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-7b352467-b4ca-430a-9306-6ba52465f8ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:00.489417 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b8124-6c6f-4525-b76a-64e991f6bd42 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:00.500012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b352467-b4ca-430a-9306-6ba52465f8ab admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:00.504600 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82217e-2673-4d9c-b66c-4d8e25b8ca32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:00.505247 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82217e-2673-4d9c-b66c-4d8e25b8ca32 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:00.505529 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82217e-2673-4d9c-b66c-4d8e25b8ca32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:00.505802 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82217e-2673-4d9c-b66c-4d8e25b8ca32 admin admin] PATH_INFO: `/v3/projects/d0cbf00874834cce9829e8ef9640e4c3/users/bdb80c613b69414283245e72f0c34e21/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:00.522957 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82217e-2673-4d9c-b66c-4d8e25b8ca32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0cbf00874834cce9829e8ef9640e4c3, user_id=bdb80c613b69414283245e72f0c34e21, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:00.525933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82217e-2673-4d9c-b66c-4d8e25b8ca32 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:00.536455 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:00.539161 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 828/1640] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:00 2022] PUT /identity/v3/projects/d0cbf00874834cce9829e8ef9640e4c3/users/bdb80c613b69414283245e72f0c34e21/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:00.539729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-7d0b8124-6c6f-4525-b76a-64e991f6bd42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cf2b2a3bc324b2e9c436efcf92d9726'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:00.540736 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-7d0b8124-6c6f-4525-b76a-64e991f6bd42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cf2b2a3bc324b2e9c436efcf92d9726'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:00.543152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82217e-2673-4d9c-b66c-4d8e25b8ca32 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:00.545017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ec82217e-2673-4d9c-b66c-4d8e25b8ca32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:00.555741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-7d0b8124-6c6f-4525-b76a-64e991f6bd42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cf2b2a3bc324b2e9c436efcf92d9726'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:00.563750 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-7d0b8124-6c6f-4525-b76a-64e991f6bd42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cf2b2a3bc324b2e9c436efcf92d9726'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:00.564741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-7d0b8124-6c6f-4525-b76a-64e991f6bd42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cf2b2a3bc324b2e9c436efcf92d9726'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:00.580816 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 813/1641] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:00 2022] DELETE /identity/v3/users/7cf2b2a3bc324b2e9c436efcf92d9726 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:00.633393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82217e-2673-4d9c-b66c-4d8e25b8ca32 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:00.693377 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:00.755044 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6a03b-0b93-4a6c-a020-e7b922764514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:00.755680 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6a03b-0b93-4a6c-a020-e7b922764514 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:00.755963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6a03b-0b93-4a6c-a020-e7b922764514 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:00.756274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6a03b-0b93-4a6c-a020-e7b922764514 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:00.756923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6a03b-0b93-4a6c-a020-e7b922764514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:00.758537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6a03b-0b93-4a6c-a020-e7b922764514 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:00.770510 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 829/1642] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:50:00.774669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b8124-6c6f-4525-b76a-64e991f6bd42 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:00.786678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b8124-6c6f-4525-b76a-64e991f6bd42 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:00.792076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cdb7e1-3f00-4860-be07-47b271a40387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:00.792762 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdb7e1-3f00-4860-be07-47b271a40387 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:00.793041 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdb7e1-3f00-4860-be07-47b271a40387 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:00.793381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdb7e1-3f00-4860-be07-47b271a40387 admin admin] PATH_INFO: `/v3/projects/d0cbf00874834cce9829e8ef9640e4c3/users/bdb80c613b69414283245e72f0c34e21/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:00.795195 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cdb7e1-3f00-4860-be07-47b271a40387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0cbf00874834cce9829e8ef9640e4c3, user_id=bdb80c613b69414283245e72f0c34e21, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:00.799164 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cdb7e1-3f00-4860-be07-47b271a40387 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:00.810388 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 814/1643] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:00 2022] PUT /identity/v3/projects/d0cbf00874834cce9829e8ef9640e4c3/users/bdb80c613b69414283245e72f0c34e21/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:00.822168 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf46a39-ceec-4980-a6d7-4ffada06b0fa None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:00.822457 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf46a39-ceec-4980-a6d7-4ffada06b0fa None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:00.822734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf46a39-ceec-4980-a6d7-4ffada06b0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:00.861259 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-1bf46a39-ceec-4980-a6d7-4ffada06b0fa None None] MFA Rules not processed for user `bdb80c613b69414283245e72f0c34e21`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:00.862476 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-1bf46a39-ceec-4980-a6d7-4ffada06b0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:00.948552 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cdb7e1-3f00-4860-be07-47b271a40387 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:00.950337 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-05cdb7e1-3f00-4860-be07-47b271a40387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:00.952485 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 830/1644] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:00 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:50:00.964860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf46a39-ceec-4980-a6d7-4ffada06b0fa None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:00.966266 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-1bf46a39-ceec-4980-a6d7-4ffada06b0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:01.027239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cdb7e1-3f00-4860-be07-47b271a40387 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:01.031779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf46a39-ceec-4980-a6d7-4ffada06b0fa None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:01.033605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62a15f-e51d-4d2e-a7a8-cf4df92d17d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:01.034245 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62a15f-e51d-4d2e-a7a8-cf4df92d17d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.034522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62a15f-e51d-4d2e-a7a8-cf4df92d17d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.034802 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62a15f-e51d-4d2e-a7a8-cf4df92d17d5 admin admin] PATH_INFO: `/v3/projects/f04b45a5dfa843aab553b3ac4306387b` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.035528 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e738f-6df1-4371-b780-28c32b952725 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:01.035911 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62a15f-e51d-4d2e-a7a8-cf4df92d17d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f04b45a5dfa843aab553b3ac4306387b)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:01.036604 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e738f-6df1-4371-b780-28c32b952725 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.036889 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e738f-6df1-4371-b780-28c32b952725 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.037263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e738f-6df1-4371-b780-28c32b952725 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.037848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62a15f-e51d-4d2e-a7a8-cf4df92d17d5 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:01.047378 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e738f-6df1-4371-b780-28c32b952725 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:01.048785 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e738f-6df1-4371-b780-28c32b952725 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:01.059928 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 831/1645] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:00 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:50:01.101537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-5f62a15f-e51d-4d2e-a7a8-cf4df92d17d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f04b45a5dfa843aab553b3ac4306387b', 'request_id': 'req-5f62a15f-e51d-4d2e-a7a8-cf4df92d17d5'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:01.109357 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-5f62a15f-e51d-4d2e-a7a8-cf4df92d17d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f04b45a5dfa843aab553b3ac4306387b', 'request_id': 'req-5f62a15f-e51d-4d2e-a7a8-cf4df92d17d5'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:01.116000 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-5f62a15f-e51d-4d2e-a7a8-cf4df92d17d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f04b45a5dfa843aab553b3ac4306387b', 'request_id': 'req-5f62a15f-e51d-4d2e-a7a8-cf4df92d17d5'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:01.128964 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-5f62a15f-e51d-4d2e-a7a8-cf4df92d17d5 admin admin] The token cache is being invalidate because project f04b45a5dfa843aab553b3ac4306387b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:01.129260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-5f62a15f-e51d-4d2e-a7a8-cf4df92d17d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:01.129938 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-5f62a15f-e51d-4d2e-a7a8-cf4df92d17d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:01.131596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 815/1646] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:00 2022] DELETE /identity/v3/projects/f04b45a5dfa843aab553b3ac4306387b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:01.137450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b169292-6215-4355-a705-779d4d937ddc None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.137765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b169292-6215-4355-a705-779d4d937ddc None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.138072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b169292-6215-4355-a705-779d4d937ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.143637 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62a15f-e51d-4d2e-a7a8-cf4df92d17d5 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:01.145337 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-5f62a15f-e51d-4d2e-a7a8-cf4df92d17d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:01.159309 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-6b169292-6215-4355-a705-779d4d937ddc None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:01.160555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-6b169292-6215-4355-a705-779d4d937ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:01.188861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:01.206741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62a15f-e51d-4d2e-a7a8-cf4df92d17d5 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:01.210448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a6c93-aa34-43a8-b580-ffa2aca4cb98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:01.211056 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a6c93-aa34-43a8-b580-ffa2aca4cb98 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.211336 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a6c93-aa34-43a8-b580-ffa2aca4cb98 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.211618 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a6c93-aa34-43a8-b580-ffa2aca4cb98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.213404 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-2a3a6c93-aa34-43a8-b580-ffa2aca4cb98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:01.214270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:01.224429 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 832/1647] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:50:01.232197 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910428e-6a1b-46b3-ba33-a07cbb136f98 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.232614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910428e-6a1b-46b3-ba33-a07cbb136f98 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.233378 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910428e-6a1b-46b3-ba33-a07cbb136f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.253426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-4910428e-6a1b-46b3-ba33-a07cbb136f98 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:01.254635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4910428e-6a1b-46b3-ba33-a07cbb136f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:01.262866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 833/1648] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:50:01.267027 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910428e-6a1b-46b3-ba33-a07cbb136f98 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:01.276395 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910428e-6a1b-46b3-ba33-a07cbb136f98 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:01.277964 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a6c93-aa34-43a8-b580-ffa2aca4cb98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:01.279412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a6c93-aa34-43a8-b580-ffa2aca4cb98 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:01.281263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84769e-e5e3-4d81-a127-81283ebdce12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:01.281853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84769e-e5e3-4d81-a127-81283ebdce12 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.282131 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84769e-e5e3-4d81-a127-81283ebdce12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.282403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84769e-e5e3-4d81-a127-81283ebdce12 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.283044 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84769e-e5e3-4d81-a127-81283ebdce12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:01.283342 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84769e-e5e3-4d81-a127-81283ebdce12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:01.284957 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84769e-e5e3-4d81-a127-81283ebdce12 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:01.288557 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 816/1649] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:01.297822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 834/1650] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:01.302455 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a6c93-aa34-43a8-b580-ffa2aca4cb98 service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:01.313490 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a6c93-aa34-43a8-b580-ffa2aca4cb98 service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:01.318288 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05c0be-345a-4f85-8e02-60a88069f35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:01.318894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05c0be-345a-4f85-8e02-60a88069f35d admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.319174 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05c0be-345a-4f85-8e02-60a88069f35d admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.319446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05c0be-345a-4f85-8e02-60a88069f35d admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.320134 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05c0be-345a-4f85-8e02-60a88069f35d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:01.322017 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05c0be-345a-4f85-8e02-60a88069f35d admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:01.350651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 817/1651] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:01 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:01.354907 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84769e-e5e3-4d81-a127-81283ebdce12 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:01.366497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84769e-e5e3-4d81-a127-81283ebdce12 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:01.429406 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93d58d-9be4-473e-8ec9-736a8e1f3439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:01.430778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93d58d-9be4-473e-8ec9-736a8e1f3439 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.430895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93d58d-9be4-473e-8ec9-736a8e1f3439 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.431330 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93d58d-9be4-473e-8ec9-736a8e1f3439 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.432829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93d58d-9be4-473e-8ec9-736a8e1f3439 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:01.435101 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93d58d-9be4-473e-8ec9-736a8e1f3439 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:01.473980 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 835/1652] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:01 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:50:01.475179 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05c0be-345a-4f85-8e02-60a88069f35d admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:01.486026 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05c0be-345a-4f85-8e02-60a88069f35d admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:01.490744 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068962d-dec6-40b0-89a0-a7317ba437e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:01.491382 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068962d-dec6-40b0-89a0-a7317ba437e5 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.491665 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068962d-dec6-40b0-89a0-a7317ba437e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.491937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068962d-dec6-40b0-89a0-a7317ba437e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.492600 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068962d-dec6-40b0-89a0-a7317ba437e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:01.494209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068962d-dec6-40b0-89a0-a7317ba437e5 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:01.505057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 818/1653] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:01.509084 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93d58d-9be4-473e-8ec9-736a8e1f3439 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:01.519518 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93d58d-9be4-473e-8ec9-736a8e1f3439 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:01.523998 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628c324-6458-4b6c-b7ed-b4f714bca313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:01.524612 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628c324-6458-4b6c-b7ed-b4f714bca313 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.524892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628c324-6458-4b6c-b7ed-b4f714bca313 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.525166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628c324-6458-4b6c-b7ed-b4f714bca313 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.525798 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628c324-6458-4b6c-b7ed-b4f714bca313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:01.527411 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628c324-6458-4b6c-b7ed-b4f714bca313 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:01.538538 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 836/1654] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:01.542684 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068962d-dec6-40b0-89a0-a7317ba437e5 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:01.552963 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068962d-dec6-40b0-89a0-a7317ba437e5 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:01.557532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb01a0-b45f-4c55-a349-592ba8e3179d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:01.558171 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb01a0-b45f-4c55-a349-592ba8e3179d admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.558447 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb01a0-b45f-4c55-a349-592ba8e3179d admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.558718 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb01a0-b45f-4c55-a349-592ba8e3179d admin admin] PATH_INFO: `/v3/projects/66bd01489a0a43dd84e5ac0d0a80aa2d/users/d7fd3fb1e7484c179a6f57d3cc6be741/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.573887 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb01a0-b45f-4c55-a349-592ba8e3179d admin admin] RBAC: Authorizing `identity:create_grant(project_id=66bd01489a0a43dd84e5ac0d0a80aa2d, user_id=d7fd3fb1e7484c179a6f57d3cc6be741, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:01.576564 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb01a0-b45f-4c55-a349-592ba8e3179d admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:01.590158 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 819/1655] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:01 2022] PUT /identity/v3/projects/66bd01489a0a43dd84e5ac0d0a80aa2d/users/d7fd3fb1e7484c179a6f57d3cc6be741/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:01.601442 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c855d34-16db-48e7-a8c5-a56e41323757 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.601734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c855d34-16db-48e7-a8c5-a56e41323757 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.602013 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c855d34-16db-48e7-a8c5-a56e41323757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.629772 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-2c855d34-16db-48e7-a8c5-a56e41323757 None None] MFA Rules not processed for user `d7fd3fb1e7484c179a6f57d3cc6be741`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:01.630986 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-2c855d34-16db-48e7-a8c5-a56e41323757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:01.694407 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 837/1656] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:01.706186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb01a0-b45f-4c55-a349-592ba8e3179d admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:01.707858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-51eb01a0-b45f-4c55-a349-592ba8e3179d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:01.765711 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb01a0-b45f-4c55-a349-592ba8e3179d admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:01.769392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883ee0a-2138-460a-84d3-06f9689a8ef0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cf9451f2ef7478589b4a4278cb03252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:01.770011 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883ee0a-2138-460a-84d3-06f9689a8ef0 service glance] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:01.770292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883ee0a-2138-460a-84d3-06f9689a8ef0 service glance] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:01.770564 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883ee0a-2138-460a-84d3-06f9689a8ef0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:01.779180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883ee0a-2138-460a-84d3-06f9689a8ef0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:01.780744 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883ee0a-2138-460a-84d3-06f9689a8ef0 service glance] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:01.789834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 820/1657] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:11.691487 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=76,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:11.785676 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=81,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:17.517176 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c855d34-16db-48e7-a8c5-a56e41323757 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:17.518725 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-2c855d34-16db-48e7-a8c5-a56e41323757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:17.601359 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c855d34-16db-48e7-a8c5-a56e41323757 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:17.605879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1a496-036d-46cf-93e4-44b840df3f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:17.606606 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1a496-036d-46cf-93e4-44b840df3f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:17.606946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1a496-036d-46cf-93e4-44b840df3f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:17.607284 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1a496-036d-46cf-93e4-44b840df3f91 admin admin] PATH_INFO: `/v3/users/027c45e46200450992eabaed48e9e716` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:17.609104 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1a496-036d-46cf-93e4-44b840df3f91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=027c45e46200450992eabaed48e9e716)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:17.611091 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1a496-036d-46cf-93e4-44b840df3f91 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:17.661086 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-38d1a496-036d-46cf-93e4-44b840df3f91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '027c45e46200450992eabaed48e9e716'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:17.661746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-38d1a496-036d-46cf-93e4-44b840df3f91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '027c45e46200450992eabaed48e9e716'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:17.674980 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-38d1a496-036d-46cf-93e4-44b840df3f91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '027c45e46200450992eabaed48e9e716'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:17.682297 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-38d1a496-036d-46cf-93e4-44b840df3f91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '027c45e46200450992eabaed48e9e716'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:17.691637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-38d1a496-036d-46cf-93e4-44b840df3f91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '027c45e46200450992eabaed48e9e716'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:17.697891 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 838/1658] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:17 2022] DELETE /identity/v3/users/027c45e46200450992eabaed48e9e716 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:17.702381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ede867-d6fa-42c0-af73-2d27f1a0cc9c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:17.702855 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ede867-d6fa-42c0-af73-2d27f1a0cc9c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:17.702957 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ede867-d6fa-42c0-af73-2d27f1a0cc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:17.722201 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-c2ede867-d6fa-42c0-af73-2d27f1a0cc9c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:17.723408 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c2ede867-d6fa-42c0-af73-2d27f1a0cc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:17.784633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 821/1659] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:17.797048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1a496-036d-46cf-93e4-44b840df3f91 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:17.798458 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-38d1a496-036d-46cf-93e4-44b840df3f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:17.862123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1a496-036d-46cf-93e4-44b840df3f91 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:17.866671 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35bea3-896f-49a5-83d8-8a657af4d2ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:17.867286 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35bea3-896f-49a5-83d8-8a657af4d2ae service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:17.867565 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35bea3-896f-49a5-83d8-8a657af4d2ae service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:17.867860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35bea3-896f-49a5-83d8-8a657af4d2ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:17.882467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35bea3-896f-49a5-83d8-8a657af4d2ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:17.883938 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35bea3-896f-49a5-83d8-8a657af4d2ae service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:17.896587 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 839/1660] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:18.212196 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ede867-d6fa-42c0-af73-2d27f1a0cc9c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:18.213803 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c2ede867-d6fa-42c0-af73-2d27f1a0cc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:18.227855 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ede867-d6fa-42c0-af73-2d27f1a0cc9c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:18.240196 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d2d8d-f1da-4e84-92f0-25b8ca3b3483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:18.240983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d2d8d-f1da-4e84-92f0-25b8ca3b3483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:18.241421 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d2d8d-f1da-4e84-92f0-25b8ca3b3483 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:18.241812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d2d8d-f1da-4e84-92f0-25b8ca3b3483 admin admin] PATH_INFO: `/v3/projects/944de69115e6480eb29bc317a4e5e5b2` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:18.243099 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d2d8d-f1da-4e84-92f0-25b8ca3b3483 admin admin] RBAC: Authorizing `identity:delete_project(project_id=944de69115e6480eb29bc317a4e5e5b2)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:18.245243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d2d8d-f1da-4e84-92f0-25b8ca3b3483 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:18.299423 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-213d2d8d-f1da-4e84-92f0-25b8ca3b3483 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '944de69115e6480eb29bc317a4e5e5b2', 'request_id': 'req-213d2d8d-f1da-4e84-92f0-25b8ca3b3483'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:18.307583 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-213d2d8d-f1da-4e84-92f0-25b8ca3b3483 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '944de69115e6480eb29bc317a4e5e5b2', 'request_id': 'req-213d2d8d-f1da-4e84-92f0-25b8ca3b3483'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:18.314069 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-213d2d8d-f1da-4e84-92f0-25b8ca3b3483 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '944de69115e6480eb29bc317a4e5e5b2', 'request_id': 'req-213d2d8d-f1da-4e84-92f0-25b8ca3b3483'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:18.325189 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-213d2d8d-f1da-4e84-92f0-25b8ca3b3483 admin admin] The token cache is being invalidate because project 944de69115e6480eb29bc317a4e5e5b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:18.325507 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-213d2d8d-f1da-4e84-92f0-25b8ca3b3483 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:18.326239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-213d2d8d-f1da-4e84-92f0-25b8ca3b3483 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:18.327875 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 822/1661] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:18 2022] DELETE /identity/v3/projects/944de69115e6480eb29bc317a4e5e5b2 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:18.332834 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0b36e-e200-4ce5-9b9a-6266da358f43 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:18.333123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0b36e-e200-4ce5-9b9a-6266da358f43 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:18.333400 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0b36e-e200-4ce5-9b9a-6266da358f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:18.353758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-90d0b36e-e200-4ce5-9b9a-6266da358f43 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:18.354985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-90d0b36e-e200-4ce5-9b9a-6266da358f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:18.372222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:18.416804 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 840/1662] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:18.425024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640f981-9737-4f5d-a2ee-9e1ad1632279 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:18.425314 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640f981-9737-4f5d-a2ee-9e1ad1632279 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:18.425600 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640f981-9737-4f5d-a2ee-9e1ad1632279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:18.445845 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-6640f981-9737-4f5d-a2ee-9e1ad1632279 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:18.447048 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-6640f981-9737-4f5d-a2ee-9e1ad1632279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:18.455095 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 823/1663] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:18.459218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0b36e-e200-4ce5-9b9a-6266da358f43 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:18.469267 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0b36e-e200-4ce5-9b9a-6266da358f43 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:18.473872 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fa9d1-ccde-45d9-9d6e-782ac1d5865c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:18.474472 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fa9d1-ccde-45d9-9d6e-782ac1d5865c admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:18.474764 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fa9d1-ccde-45d9-9d6e-782ac1d5865c admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:18.475042 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fa9d1-ccde-45d9-9d6e-782ac1d5865c admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:18.475683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77fa9d1-ccde-45d9-9d6e-782ac1d5865c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:18.475989 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77fa9d1-ccde-45d9-9d6e-782ac1d5865c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:18.477623 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77fa9d1-ccde-45d9-9d6e-782ac1d5865c admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:18.489878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 841/1664] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:18.494156 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640f981-9737-4f5d-a2ee-9e1ad1632279 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:18.503984 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640f981-9737-4f5d-a2ee-9e1ad1632279 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:18.508720 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362cc34-6dd7-4c27-a232-e78901c53d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:18.509324 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362cc34-6dd7-4c27-a232-e78901c53d36 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:18.509609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362cc34-6dd7-4c27-a232-e78901c53d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:18.509895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362cc34-6dd7-4c27-a232-e78901c53d36 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:18.510538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362cc34-6dd7-4c27-a232-e78901c53d36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:18.512759 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362cc34-6dd7-4c27-a232-e78901c53d36 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:18.545926 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 824/1665] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:18 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:50:18.549811 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fa9d1-ccde-45d9-9d6e-782ac1d5865c admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:18.561970 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fa9d1-ccde-45d9-9d6e-782ac1d5865c admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:18.648140 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53555ba4-0edf-4f0e-b7d7-fa0a1cbcb7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:18.648994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53555ba4-0edf-4f0e-b7d7-fa0a1cbcb7c1 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:18.649282 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53555ba4-0edf-4f0e-b7d7-fa0a1cbcb7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:18.649650 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53555ba4-0edf-4f0e-b7d7-fa0a1cbcb7c1 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:18.650371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53555ba4-0edf-4f0e-b7d7-fa0a1cbcb7c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:18.652528 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53555ba4-0edf-4f0e-b7d7-fa0a1cbcb7c1 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:18.679199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 842/1666] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:18 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:18.682445 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362cc34-6dd7-4c27-a232-e78901c53d36 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:18.694125 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362cc34-6dd7-4c27-a232-e78901c53d36 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:18.698908 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2641c-5d71-4285-86ad-3027f1a9f422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:18.699519 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2641c-5d71-4285-86ad-3027f1a9f422 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:18.699796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2641c-5d71-4285-86ad-3027f1a9f422 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:18.700065 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2641c-5d71-4285-86ad-3027f1a9f422 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:18.700726 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c2641c-5d71-4285-86ad-3027f1a9f422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:18.702711 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c2641c-5d71-4285-86ad-3027f1a9f422 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:18.713040 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 825/1667] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:18.717206 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53555ba4-0edf-4f0e-b7d7-fa0a1cbcb7c1 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:18.727291 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53555ba4-0edf-4f0e-b7d7-fa0a1cbcb7c1 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:18.732073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bccae-a0eb-4e8e-9b08-a950c6bb4036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:18.733215 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bccae-a0eb-4e8e-9b08-a950c6bb4036 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:18.733535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bccae-a0eb-4e8e-9b08-a950c6bb4036 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:18.733909 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bccae-a0eb-4e8e-9b08-a950c6bb4036 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:18.734642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bccae-a0eb-4e8e-9b08-a950c6bb4036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:18.736348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bccae-a0eb-4e8e-9b08-a950c6bb4036 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:18.750103 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 843/1668] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:18.754751 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2641c-5d71-4285-86ad-3027f1a9f422 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:18.767508 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2641c-5d71-4285-86ad-3027f1a9f422 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:18.772946 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7784576-bbe5-4e72-a92b-2497376fa3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:18.773612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7784576-bbe5-4e72-a92b-2497376fa3ad admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:18.773897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7784576-bbe5-4e72-a92b-2497376fa3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:18.774175 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7784576-bbe5-4e72-a92b-2497376fa3ad admin admin] PATH_INFO: `/v3/projects/ccd8e44dabf34f32a1c2610edf90c83f/users/284b39b9844844e182b2ef3542a6abf2/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:18.790888 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7784576-bbe5-4e72-a92b-2497376fa3ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccd8e44dabf34f32a1c2610edf90c83f, user_id=284b39b9844844e182b2ef3542a6abf2, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:18.793597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7784576-bbe5-4e72-a92b-2497376fa3ad admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:18.807992 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 826/1669] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:18 2022] PUT /identity/v3/projects/ccd8e44dabf34f32a1c2610edf90c83f/users/284b39b9844844e182b2ef3542a6abf2/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:18.821789 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6693ba6-f3be-474e-b923-c9eda597332e None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:18.822235 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6693ba6-f3be-474e-b923-c9eda597332e None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:18.822682 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6693ba6-f3be-474e-b923-c9eda597332e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:18.857324 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-b6693ba6-f3be-474e-b923-c9eda597332e None None] MFA Rules not processed for user `284b39b9844844e182b2ef3542a6abf2`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:18.859513 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-b6693ba6-f3be-474e-b923-c9eda597332e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:18.947443 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 844/1670] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:18.959090 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7784576-bbe5-4e72-a92b-2497376fa3ad admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:18.962517 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a7784576-bbe5-4e72-a92b-2497376fa3ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:18.993401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:19.034504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7784576-bbe5-4e72-a92b-2497376fa3ad admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:19.038519 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8ebbe-bcef-4214-89ad-cd657e01dc82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:19.039122 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8ebbe-bcef-4214-89ad-cd657e01dc82 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:19.039397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8ebbe-bcef-4214-89ad-cd657e01dc82 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:19.039665 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8ebbe-bcef-4214-89ad-cd657e01dc82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:19.049282 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8ebbe-bcef-4214-89ad-cd657e01dc82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:19.050713 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8ebbe-bcef-4214-89ad-cd657e01dc82 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:19.059345 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 827/1671] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:19.083859 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6693ba6-f3be-474e-b923-c9eda597332e None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:19.103308 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:19.147618 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6693ba6-f3be-474e-b923-c9eda597332e None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:19.152045 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65aa09d-9675-429e-89c3-a4d070676594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:19.152701 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65aa09d-9675-429e-89c3-a4d070676594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:19.152977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65aa09d-9675-429e-89c3-a4d070676594 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:19.154103 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65aa09d-9675-429e-89c3-a4d070676594 admin admin] PATH_INFO: `/v3/users/284b39b9844844e182b2ef3542a6abf2` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:19.154346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65aa09d-9675-429e-89c3-a4d070676594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=284b39b9844844e182b2ef3542a6abf2)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:19.156999 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65aa09d-9675-429e-89c3-a4d070676594 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:19.202980 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a65aa09d-9675-429e-89c3-a4d070676594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '284b39b9844844e182b2ef3542a6abf2'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:19.203839 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a65aa09d-9675-429e-89c3-a4d070676594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '284b39b9844844e182b2ef3542a6abf2'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:19.216009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a65aa09d-9675-429e-89c3-a4d070676594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '284b39b9844844e182b2ef3542a6abf2'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:19.223686 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a65aa09d-9675-429e-89c3-a4d070676594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '284b39b9844844e182b2ef3542a6abf2'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:19.231754 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a65aa09d-9675-429e-89c3-a4d070676594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '284b39b9844844e182b2ef3542a6abf2'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:19.238072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 845/1672] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:19 2022] DELETE /identity/v3/users/284b39b9844844e182b2ef3542a6abf2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:19.243472 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6b829-0485-480e-b462-7d75ffc17a84 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:19.243826 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6b829-0485-480e-b462-7d75ffc17a84 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:19.244272 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6b829-0485-480e-b462-7d75ffc17a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:19.270434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-94d6b829-0485-480e-b462-7d75ffc17a84 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:19.271693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-94d6b829-0485-480e-b462-7d75ffc17a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:19.358131 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 828/1673] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:19.368005 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65aa09d-9675-429e-89c3-a4d070676594 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:19.369443 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a65aa09d-9675-429e-89c3-a4d070676594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:19.429348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65aa09d-9675-429e-89c3-a4d070676594 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:19.433263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f19be-32b7-4a4e-8928-20eb28d1a9a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:19.433884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f19be-32b7-4a4e-8928-20eb28d1a9a6 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:19.434166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f19be-32b7-4a4e-8928-20eb28d1a9a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:19.434440 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f19be-32b7-4a4e-8928-20eb28d1a9a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:19.443888 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f19be-32b7-4a4e-8928-20eb28d1a9a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:19.445352 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f19be-32b7-4a4e-8928-20eb28d1a9a6 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:19.455368 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 846/1674] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:50:19.741409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6b829-0485-480e-b462-7d75ffc17a84 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:19.743104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-94d6b829-0485-480e-b462-7d75ffc17a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:19.759690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6b829-0485-480e-b462-7d75ffc17a84 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:19.764913 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0598c417-f489-4400-8a6a-7fc092f508a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:19.765567 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598c417-f489-4400-8a6a-7fc092f508a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:19.765845 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598c417-f489-4400-8a6a-7fc092f508a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:19.766112 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598c417-f489-4400-8a6a-7fc092f508a0 admin admin] PATH_INFO: `/v3/projects/ccd8e44dabf34f32a1c2610edf90c83f` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:19.767425 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0598c417-f489-4400-8a6a-7fc092f508a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccd8e44dabf34f32a1c2610edf90c83f)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:19.769665 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0598c417-f489-4400-8a6a-7fc092f508a0 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:19.816152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-0598c417-f489-4400-8a6a-7fc092f508a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccd8e44dabf34f32a1c2610edf90c83f', 'request_id': 'req-0598c417-f489-4400-8a6a-7fc092f508a0'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:19.823252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-0598c417-f489-4400-8a6a-7fc092f508a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccd8e44dabf34f32a1c2610edf90c83f', 'request_id': 'req-0598c417-f489-4400-8a6a-7fc092f508a0'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:19.830334 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-0598c417-f489-4400-8a6a-7fc092f508a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccd8e44dabf34f32a1c2610edf90c83f', 'request_id': 'req-0598c417-f489-4400-8a6a-7fc092f508a0'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:19.840888 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-0598c417-f489-4400-8a6a-7fc092f508a0 admin admin] The token cache is being invalidate because project ccd8e44dabf34f32a1c2610edf90c83f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:19.841182 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-0598c417-f489-4400-8a6a-7fc092f508a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:19.841828 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-0598c417-f489-4400-8a6a-7fc092f508a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:19.843440 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 829/1675] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:19 2022] DELETE /identity/v3/projects/ccd8e44dabf34f32a1c2610edf90c83f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:19.848309 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5562139-09d9-4653-9fd4-3d534c9324be None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:19.848799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5562139-09d9-4653-9fd4-3d534c9324be None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:19.848891 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5562139-09d9-4653-9fd4-3d534c9324be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:19.868720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-a5562139-09d9-4653-9fd4-3d534c9324be None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:19.869919 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a5562139-09d9-4653-9fd4-3d534c9324be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:19.906739 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:19.934424 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 847/1676] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:50:19.943217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91171636-61c7-4fcb-ba44-3a3da7d4ad0c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:19.943508 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91171636-61c7-4fcb-ba44-3a3da7d4ad0c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:19.943791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91171636-61c7-4fcb-ba44-3a3da7d4ad0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:19.964459 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-91171636-61c7-4fcb-ba44-3a3da7d4ad0c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:19.966017 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-91171636-61c7-4fcb-ba44-3a3da7d4ad0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:19.974594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 830/1677] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:19.978822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5562139-09d9-4653-9fd4-3d534c9324be None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:19.989209 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5562139-09d9-4653-9fd4-3d534c9324be None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:19.994866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477c720-4766-48d2-8c2e-387d5701cc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:19.995469 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477c720-4766-48d2-8c2e-387d5701cc8c admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:19.995751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477c720-4766-48d2-8c2e-387d5701cc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:19.996024 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477c720-4766-48d2-8c2e-387d5701cc8c admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:19.996734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477c720-4766-48d2-8c2e-387d5701cc8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:19.997036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477c720-4766-48d2-8c2e-387d5701cc8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:19.999494 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477c720-4766-48d2-8c2e-387d5701cc8c admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:20.015594 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 848/1678] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:20.016645 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91171636-61c7-4fcb-ba44-3a3da7d4ad0c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:20.026102 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91171636-61c7-4fcb-ba44-3a3da7d4ad0c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:20.031685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c3a8a-a4bb-4cd7-8ae4-4e1f2300462f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:20.032471 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c3a8a-a4bb-4cd7-8ae4-4e1f2300462f admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:20.032702 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c3a8a-a4bb-4cd7-8ae4-4e1f2300462f admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:20.032886 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c3a8a-a4bb-4cd7-8ae4-4e1f2300462f admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:20.033530 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c3a8a-a4bb-4cd7-8ae4-4e1f2300462f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:20.035467 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c3a8a-a4bb-4cd7-8ae4-4e1f2300462f admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:20.070404 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 831/1679] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:20 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:20.085387 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477c720-4766-48d2-8c2e-387d5701cc8c admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:20.103563 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477c720-4766-48d2-8c2e-387d5701cc8c admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:20.183543 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cb5c7-45b5-4933-a758-6e2ae281ddba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:20.184362 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cb5c7-45b5-4933-a758-6e2ae281ddba admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:20.184650 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cb5c7-45b5-4933-a758-6e2ae281ddba admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:20.184928 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69cb5c7-45b5-4933-a758-6e2ae281ddba admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:20.185617 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cb5c7-45b5-4933-a758-6e2ae281ddba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:20.188557 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69cb5c7-45b5-4933-a758-6e2ae281ddba admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:20.211043 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 849/1680] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:20 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:20.215505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c3a8a-a4bb-4cd7-8ae4-4e1f2300462f admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:20.226394 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c3a8a-a4bb-4cd7-8ae4-4e1f2300462f admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:20.234064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50468d9d-4bd7-4427-be43-f5d69420691f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:20.234672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50468d9d-4bd7-4427-be43-f5d69420691f admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:20.234953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50468d9d-4bd7-4427-be43-f5d69420691f admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:20.235786 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50468d9d-4bd7-4427-be43-f5d69420691f admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:20.236499 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50468d9d-4bd7-4427-be43-f5d69420691f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:20.238163 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50468d9d-4bd7-4427-be43-f5d69420691f admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:20.249921 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 832/1681] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:50:20.253999 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cb5c7-45b5-4933-a758-6e2ae281ddba admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:20.263870 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69cb5c7-45b5-4933-a758-6e2ae281ddba admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:20.268887 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b945aa-aa5a-4789-aa9c-d31e2a53dc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:20.269497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b945aa-aa5a-4789-aa9c-d31e2a53dc10 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:20.269792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b945aa-aa5a-4789-aa9c-d31e2a53dc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:20.270051 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b945aa-aa5a-4789-aa9c-d31e2a53dc10 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:20.270695 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b945aa-aa5a-4789-aa9c-d31e2a53dc10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:20.272389 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b945aa-aa5a-4789-aa9c-d31e2a53dc10 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:20.283107 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 850/1682] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:20.287061 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50468d9d-4bd7-4427-be43-f5d69420691f admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:20.298017 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50468d9d-4bd7-4427-be43-f5d69420691f admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:20.302844 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6550ff-979c-479f-a127-d498f97b42f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:20.303423 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6550ff-979c-479f-a127-d498f97b42f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:20.303746 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6550ff-979c-479f-a127-d498f97b42f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:20.304000 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6550ff-979c-479f-a127-d498f97b42f7 admin admin] PATH_INFO: `/v3/projects/476eba79ed9e41d18d47762b95a27ebd/users/fce5acf54e0047e6864df235eb560a0c/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:20.308449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:20.324344 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6550ff-979c-479f-a127-d498f97b42f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=476eba79ed9e41d18d47762b95a27ebd, user_id=fce5acf54e0047e6864df235eb560a0c, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:20.327387 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6550ff-979c-479f-a127-d498f97b42f7 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:20.341331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 833/1683] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:20 2022] PUT /identity/v3/projects/476eba79ed9e41d18d47762b95a27ebd/users/fce5acf54e0047e6864df235eb560a0c/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:20.352637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d8e2d-4f73-4f69-8161-2ee28c535f46 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:20.352984 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d8e2d-4f73-4f69-8161-2ee28c535f46 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:20.353258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d8e2d-4f73-4f69-8161-2ee28c535f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:20.382692 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-765d8e2d-4f73-4f69-8161-2ee28c535f46 None None] MFA Rules not processed for user `fce5acf54e0047e6864df235eb560a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:20.384013 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-765d8e2d-4f73-4f69-8161-2ee28c535f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:20.468230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 851/1684] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:20.478923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6550ff-979c-479f-a127-d498f97b42f7 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:20.480766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-7e6550ff-979c-479f-a127-d498f97b42f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:20.547234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6550ff-979c-479f-a127-d498f97b42f7 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:20.550978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606c78e-57ff-41de-a494-e6c2b8b69531 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:20.551624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606c78e-57ff-41de-a494-e6c2b8b69531 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:20.551999 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606c78e-57ff-41de-a494-e6c2b8b69531 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:20.552286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606c78e-57ff-41de-a494-e6c2b8b69531 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:20.564704 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606c78e-57ff-41de-a494-e6c2b8b69531 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:20.566221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606c78e-57ff-41de-a494-e6c2b8b69531 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:20.575854 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 834/1685] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:25.356714 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d8e2d-4f73-4f69-8161-2ee28c535f46 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:25.358654 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-765d8e2d-4f73-4f69-8161-2ee28c535f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:25.410287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:25.437412 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d8e2d-4f73-4f69-8161-2ee28c535f46 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:25.442201 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a865f6-15a0-4b6b-9367-3c321ed0e2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:25.442832 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a865f6-15a0-4b6b-9367-3c321ed0e2fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:25.443110 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a865f6-15a0-4b6b-9367-3c321ed0e2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:25.443381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a865f6-15a0-4b6b-9367-3c321ed0e2fe admin admin] PATH_INFO: `/v3/users/269a69f4f0694ed5b27a1fad9d222691` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:25.444621 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a865f6-15a0-4b6b-9367-3c321ed0e2fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=269a69f4f0694ed5b27a1fad9d222691)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:25.446704 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a865f6-15a0-4b6b-9367-3c321ed0e2fe admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:25.498453 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-93a865f6-15a0-4b6b-9367-3c321ed0e2fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269a69f4f0694ed5b27a1fad9d222691'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:25.499221 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-93a865f6-15a0-4b6b-9367-3c321ed0e2fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269a69f4f0694ed5b27a1fad9d222691'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:25.512683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-93a865f6-15a0-4b6b-9367-3c321ed0e2fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269a69f4f0694ed5b27a1fad9d222691'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:25.520755 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-93a865f6-15a0-4b6b-9367-3c321ed0e2fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269a69f4f0694ed5b27a1fad9d222691'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:25.529484 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-93a865f6-15a0-4b6b-9367-3c321ed0e2fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269a69f4f0694ed5b27a1fad9d222691'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:25.536829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 852/1686] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:25 2022] DELETE /identity/v3/users/269a69f4f0694ed5b27a1fad9d222691 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:25.541836 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1658b7-91fd-4eac-919b-2c7260ee3921 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:25.542314 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1658b7-91fd-4eac-919b-2c7260ee3921 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:25.542314 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1658b7-91fd-4eac-919b-2c7260ee3921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:25.566608 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-4f1658b7-91fd-4eac-919b-2c7260ee3921 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:25.569153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4f1658b7-91fd-4eac-919b-2c7260ee3921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:25.712745 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 835/1687] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:25.723243 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a865f6-15a0-4b6b-9367-3c321ed0e2fe admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:25.724730 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-93a865f6-15a0-4b6b-9367-3c321ed0e2fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:25.797030 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a865f6-15a0-4b6b-9367-3c321ed0e2fe admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:25.800693 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1b27e-4200-4f4c-a852-56de935eed19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:25.801303 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1b27e-4200-4f4c-a852-56de935eed19 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:25.801583 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1b27e-4200-4f4c-a852-56de935eed19 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:25.801849 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1b27e-4200-4f4c-a852-56de935eed19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:25.812285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b1b27e-4200-4f4c-a852-56de935eed19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:25.813740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b1b27e-4200-4f4c-a852-56de935eed19 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:25.823394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 853/1688] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:26.112376 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1658b7-91fd-4eac-919b-2c7260ee3921 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:26.113896 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4f1658b7-91fd-4eac-919b-2c7260ee3921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:26.128321 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1658b7-91fd-4eac-919b-2c7260ee3921 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:26.135090 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4d14c-cd8b-4707-8399-7c3dd2866715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:26.135753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4d14c-cd8b-4707-8399-7c3dd2866715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:26.136032 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4d14c-cd8b-4707-8399-7c3dd2866715 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:26.136422 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4d14c-cd8b-4707-8399-7c3dd2866715 admin admin] PATH_INFO: `/v3/projects/35a58dbd858c4153b732ae1b5edeb8de` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:26.137859 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4d14c-cd8b-4707-8399-7c3dd2866715 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35a58dbd858c4153b732ae1b5edeb8de)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:26.141607 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4d14c-cd8b-4707-8399-7c3dd2866715 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:26.191209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6ea4d14c-cd8b-4707-8399-7c3dd2866715 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35a58dbd858c4153b732ae1b5edeb8de', 'request_id': 'req-6ea4d14c-cd8b-4707-8399-7c3dd2866715'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:26.202748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6ea4d14c-cd8b-4707-8399-7c3dd2866715 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35a58dbd858c4153b732ae1b5edeb8de', 'request_id': 'req-6ea4d14c-cd8b-4707-8399-7c3dd2866715'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:26.211646 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6ea4d14c-cd8b-4707-8399-7c3dd2866715 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35a58dbd858c4153b732ae1b5edeb8de', 'request_id': 'req-6ea4d14c-cd8b-4707-8399-7c3dd2866715'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:26.224929 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6ea4d14c-cd8b-4707-8399-7c3dd2866715 admin admin] The token cache is being invalidate because project 35a58dbd858c4153b732ae1b5edeb8de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:26.225214 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6ea4d14c-cd8b-4707-8399-7c3dd2866715 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:26.226348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6ea4d14c-cd8b-4707-8399-7c3dd2866715 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:26.228180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 836/1689] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:26 2022] DELETE /identity/v3/projects/35a58dbd858c4153b732ae1b5edeb8de => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:26.232032 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1b27e-4200-4f4c-a852-56de935eed19 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:26.243327 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1b27e-4200-4f4c-a852-56de935eed19 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:26.320603 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:26.328999 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9a6a1-eacf-4837-8d35-ff5dbbc66f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:26.329988 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9a6a1-eacf-4837-8d35-ff5dbbc66f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:26.332382 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9a6a1-eacf-4837-8d35-ff5dbbc66f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:26.333037 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9a6a1-eacf-4837-8d35-ff5dbbc66f77 admin admin] PATH_INFO: `/v3/users/bdb80c613b69414283245e72f0c34e21` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:26.334511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9a6a1-eacf-4837-8d35-ff5dbbc66f77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdb80c613b69414283245e72f0c34e21)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:26.336600 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9a6a1-eacf-4837-8d35-ff5dbbc66f77 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:26.390595 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d0b9a6a1-eacf-4837-8d35-ff5dbbc66f77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdb80c613b69414283245e72f0c34e21'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:26.391286 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d0b9a6a1-eacf-4837-8d35-ff5dbbc66f77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdb80c613b69414283245e72f0c34e21'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:26.410670 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d0b9a6a1-eacf-4837-8d35-ff5dbbc66f77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdb80c613b69414283245e72f0c34e21'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:26.419029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d0b9a6a1-eacf-4837-8d35-ff5dbbc66f77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdb80c613b69414283245e72f0c34e21'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:26.432516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d0b9a6a1-eacf-4837-8d35-ff5dbbc66f77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdb80c613b69414283245e72f0c34e21'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:26.457721 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 854/1690] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:26 2022] DELETE /identity/v3/users/bdb80c613b69414283245e72f0c34e21 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:26.830367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4d14c-cd8b-4707-8399-7c3dd2866715 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:26.832260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-6ea4d14c-cd8b-4707-8399-7c3dd2866715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:26.851478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4d14c-cd8b-4707-8399-7c3dd2866715 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:26.864279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:26.937877 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb6bae-dddb-4523-ac91-692262831a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:26.938598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb6bae-dddb-4523-ac91-692262831a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:26.938970 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb6bae-dddb-4523-ac91-692262831a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:26.939332 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb6bae-dddb-4523-ac91-692262831a8c admin admin] PATH_INFO: `/v3/projects/d0cbf00874834cce9829e8ef9640e4c3` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:26.940788 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb6bae-dddb-4523-ac91-692262831a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0cbf00874834cce9829e8ef9640e4c3)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:26.942920 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb6bae-dddb-4523-ac91-692262831a8c admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:26.993719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-bfcb6bae-dddb-4523-ac91-692262831a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0cbf00874834cce9829e8ef9640e4c3', 'request_id': 'req-bfcb6bae-dddb-4523-ac91-692262831a8c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:27.003200 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-bfcb6bae-dddb-4523-ac91-692262831a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0cbf00874834cce9829e8ef9640e4c3', 'request_id': 'req-bfcb6bae-dddb-4523-ac91-692262831a8c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:27.010128 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-bfcb6bae-dddb-4523-ac91-692262831a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0cbf00874834cce9829e8ef9640e4c3', 'request_id': 'req-bfcb6bae-dddb-4523-ac91-692262831a8c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:27.022681 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-bfcb6bae-dddb-4523-ac91-692262831a8c admin admin] The token cache is being invalidate because project d0cbf00874834cce9829e8ef9640e4c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:27.023050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-bfcb6bae-dddb-4523-ac91-692262831a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:27.023962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-bfcb6bae-dddb-4523-ac91-692262831a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:27.026587 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 837/1691] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:26 2022] DELETE /identity/v3/projects/d0cbf00874834cce9829e8ef9640e4c3 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:27.032036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c0a83-edc8-419b-a2cf-8f0cf1eee520 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:27.032355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c0a83-edc8-419b-a2cf-8f0cf1eee520 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:27.032636 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c0a83-edc8-419b-a2cf-8f0cf1eee520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:27.055947 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-ba9c0a83-edc8-419b-a2cf-8f0cf1eee520 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:27.057209 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ba9c0a83-edc8-419b-a2cf-8f0cf1eee520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:27.128781 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 855/1692] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:50:27.131631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e96b5a-1ddf-4762-8c7d-47e20b6a4695 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:27.131917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e96b5a-1ddf-4762-8c7d-47e20b6a4695 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:27.132218 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e96b5a-1ddf-4762-8c7d-47e20b6a4695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:27.153204 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-77e96b5a-1ddf-4762-8c7d-47e20b6a4695 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:27.154432 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-77e96b5a-1ddf-4762-8c7d-47e20b6a4695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:27.166960 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 838/1693] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:27.171637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c0a83-edc8-419b-a2cf-8f0cf1eee520 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:27.184408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c0a83-edc8-419b-a2cf-8f0cf1eee520 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:27.192307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907b348-93e9-4a10-bebf-6fb5c32139f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:27.193312 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907b348-93e9-4a10-bebf-6fb5c32139f4 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:27.194151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907b348-93e9-4a10-bebf-6fb5c32139f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:27.194588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907b348-93e9-4a10-bebf-6fb5c32139f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:27.195535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907b348-93e9-4a10-bebf-6fb5c32139f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:27.196369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907b348-93e9-4a10-bebf-6fb5c32139f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:27.198310 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907b348-93e9-4a10-bebf-6fb5c32139f4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:27.214429 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 856/1694] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:27.219994 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e96b5a-1ddf-4762-8c7d-47e20b6a4695 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:27.229413 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e96b5a-1ddf-4762-8c7d-47e20b6a4695 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:27.235405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377e9f3-26a9-41f9-b164-6d2ba9db7847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:27.236495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377e9f3-26a9-41f9-b164-6d2ba9db7847 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:27.236850 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377e9f3-26a9-41f9-b164-6d2ba9db7847 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:27.237198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377e9f3-26a9-41f9-b164-6d2ba9db7847 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:27.237904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f377e9f3-26a9-41f9-b164-6d2ba9db7847 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:27.239941 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f377e9f3-26a9-41f9-b164-6d2ba9db7847 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:27.272265 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 839/1695] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:27 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:27.276487 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907b348-93e9-4a10-bebf-6fb5c32139f4 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:27.288472 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907b348-93e9-4a10-bebf-6fb5c32139f4 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:27.349270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84405954-d96b-4084-86cb-8bdb1972f2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:27.349891 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84405954-d96b-4084-86cb-8bdb1972f2ef admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:27.350175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84405954-d96b-4084-86cb-8bdb1972f2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:27.350451 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84405954-d96b-4084-86cb-8bdb1972f2ef admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:27.351096 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84405954-d96b-4084-86cb-8bdb1972f2ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:27.353584 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84405954-d96b-4084-86cb-8bdb1972f2ef admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:27.366281 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:27.386127 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 857/1696] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:27 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:27.386819 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377e9f3-26a9-41f9-b164-6d2ba9db7847 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:27.400060 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377e9f3-26a9-41f9-b164-6d2ba9db7847 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:27.405879 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecb00f-99b6-4990-aa2a-5a0b484f51cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:27.406507 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecb00f-99b6-4990-aa2a-5a0b484f51cb admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:27.406783 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecb00f-99b6-4990-aa2a-5a0b484f51cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:27.407050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecb00f-99b6-4990-aa2a-5a0b484f51cb admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:27.407686 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ecb00f-99b6-4990-aa2a-5a0b484f51cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:27.409319 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ecb00f-99b6-4990-aa2a-5a0b484f51cb admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:27.419437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 840/1697] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:50:27.424193 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84405954-d96b-4084-86cb-8bdb1972f2ef admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:27.434291 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84405954-d96b-4084-86cb-8bdb1972f2ef admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:27.439311 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879fbae-80d8-41a4-9d7f-6341fa883857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:27.440229 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879fbae-80d8-41a4-9d7f-6341fa883857 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:27.440516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879fbae-80d8-41a4-9d7f-6341fa883857 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:27.440793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879fbae-80d8-41a4-9d7f-6341fa883857 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:27.441432 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879fbae-80d8-41a4-9d7f-6341fa883857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:27.443025 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879fbae-80d8-41a4-9d7f-6341fa883857 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:27.454990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 858/1698] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:27.459016 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecb00f-99b6-4990-aa2a-5a0b484f51cb admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:27.471916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ecb00f-99b6-4990-aa2a-5a0b484f51cb admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:27.477003 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c58c2-d078-4bd6-b7e9-ff7a722facc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:27.477700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c58c2-d078-4bd6-b7e9-ff7a722facc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:27.477984 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c58c2-d078-4bd6-b7e9-ff7a722facc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:27.478258 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c58c2-d078-4bd6-b7e9-ff7a722facc7 admin admin] PATH_INFO: `/v3/projects/3a11b99fca8349fd868672bdc2d76193/users/070faaa05a1a453f9bd6b0324fa168f1/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:27.494450 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c58c2-d078-4bd6-b7e9-ff7a722facc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a11b99fca8349fd868672bdc2d76193, user_id=070faaa05a1a453f9bd6b0324fa168f1, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:27.499493 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c58c2-d078-4bd6-b7e9-ff7a722facc7 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:27.521951 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 841/1699] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:27 2022] PUT /identity/v3/projects/3a11b99fca8349fd868672bdc2d76193/users/070faaa05a1a453f9bd6b0324fa168f1/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:27.533461 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16200a2b-da13-4bc7-a78e-b76e2a091713 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:27.533754 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16200a2b-da13-4bc7-a78e-b76e2a091713 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:27.534030 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16200a2b-da13-4bc7-a78e-b76e2a091713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:27.564121 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-16200a2b-da13-4bc7-a78e-b76e2a091713 None None] MFA Rules not processed for user `070faaa05a1a453f9bd6b0324fa168f1`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:27.565765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-16200a2b-da13-4bc7-a78e-b76e2a091713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:27.637139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 859/1700] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:27.648233 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c58c2-d078-4bd6-b7e9-ff7a722facc7 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:27.703093 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:27.725311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c58c2-d078-4bd6-b7e9-ff7a722facc7 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:27.730769 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e39c8-abd8-4216-8a24-1e24c6c08bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:27.731386 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e39c8-abd8-4216-8a24-1e24c6c08bde admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:27.731672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e39c8-abd8-4216-8a24-1e24c6c08bde admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:27.731952 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e39c8-abd8-4216-8a24-1e24c6c08bde admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:27.733404 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e39c8-abd8-4216-8a24-1e24c6c08bde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:27.734946 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e39c8-abd8-4216-8a24-1e24c6c08bde admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:27.805096 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 842/1701] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:27 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:50:27.807213 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16200a2b-da13-4bc7-a78e-b76e2a091713 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:27.830829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16200a2b-da13-4bc7-a78e-b76e2a091713 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:27.897120 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c6a7f-887d-48b1-8e4b-37bcd906d278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:27.898162 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c6a7f-887d-48b1-8e4b-37bcd906d278 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:27.898593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c6a7f-887d-48b1-8e4b-37bcd906d278 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:27.899035 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c6a7f-887d-48b1-8e4b-37bcd906d278 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:27.899700 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c6a7f-887d-48b1-8e4b-37bcd906d278 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:27.901726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c6a7f-887d-48b1-8e4b-37bcd906d278 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:27.927961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 860/1702] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:27 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:50:27.929147 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e39c8-abd8-4216-8a24-1e24c6c08bde admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:27.940701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e39c8-abd8-4216-8a24-1e24c6c08bde admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:27.945500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386de37a-6ed2-4a5e-a0c7-90026b4943dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:27.946143 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386de37a-6ed2-4a5e-a0c7-90026b4943dc admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:27.946430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386de37a-6ed2-4a5e-a0c7-90026b4943dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:27.946709 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386de37a-6ed2-4a5e-a0c7-90026b4943dc admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:27.947351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386de37a-6ed2-4a5e-a0c7-90026b4943dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:27.949028 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386de37a-6ed2-4a5e-a0c7-90026b4943dc admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:27.960960 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 843/1703] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:27.965055 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c6a7f-887d-48b1-8e4b-37bcd906d278 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:27.976231 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c6a7f-887d-48b1-8e4b-37bcd906d278 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:27.981033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0de37-90ef-48ea-8edd-409236dbfe12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:27.981764 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0de37-90ef-48ea-8edd-409236dbfe12 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:27.982186 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0de37-90ef-48ea-8edd-409236dbfe12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:27.982537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0de37-90ef-48ea-8edd-409236dbfe12 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:27.983320 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0de37-90ef-48ea-8edd-409236dbfe12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:27.985102 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0de37-90ef-48ea-8edd-409236dbfe12 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:27.996221 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 861/1704] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:50:28.004980 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386de37a-6ed2-4a5e-a0c7-90026b4943dc admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:28.020017 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386de37a-6ed2-4a5e-a0c7-90026b4943dc admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:28.026729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79055f5-8091-48dd-954c-44b223b41f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:28.027403 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79055f5-8091-48dd-954c-44b223b41f06 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:28.027711 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79055f5-8091-48dd-954c-44b223b41f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:28.027960 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79055f5-8091-48dd-954c-44b223b41f06 admin admin] PATH_INFO: `/v3/projects/278005942db84b23a399005c0d170389/users/154d186c97fa4073b3e141765b0f6cb0/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:28.050707 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79055f5-8091-48dd-954c-44b223b41f06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=278005942db84b23a399005c0d170389, user_id=154d186c97fa4073b3e141765b0f6cb0, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:28.053889 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79055f5-8091-48dd-954c-44b223b41f06 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:28.075606 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 844/1705] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:28 2022] PUT /identity/v3/projects/278005942db84b23a399005c0d170389/users/154d186c97fa4073b3e141765b0f6cb0/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:28.084675 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f677a63-9e13-4fdb-9b17-bf36e463e77b None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:28.085163 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f677a63-9e13-4fdb-9b17-bf36e463e77b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:28.085571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f677a63-9e13-4fdb-9b17-bf36e463e77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:28.116600 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-2f677a63-9e13-4fdb-9b17-bf36e463e77b None None] MFA Rules not processed for user `154d186c97fa4073b3e141765b0f6cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:28.117822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-2f677a63-9e13-4fdb-9b17-bf36e463e77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:28.166395 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:28.204818 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 862/1706] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:28.209415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79055f5-8091-48dd-954c-44b223b41f06 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:28.313275 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79055f5-8091-48dd-954c-44b223b41f06 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:28.321496 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e4f57-6992-42c4-be58-ce10b12110e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:28.322594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e4f57-6992-42c4-be58-ce10b12110e0 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:28.322972 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e4f57-6992-42c4-be58-ce10b12110e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:28.323388 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e4f57-6992-42c4-be58-ce10b12110e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:28.324295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e4f57-6992-42c4-be58-ce10b12110e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:28.328196 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e4f57-6992-42c4-be58-ce10b12110e0 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:28.363917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 845/1707] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:28 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:50:28.367955 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f677a63-9e13-4fdb-9b17-bf36e463e77b None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:28.382291 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f677a63-9e13-4fdb-9b17-bf36e463e77b None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:28.447200 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468c4df-cd02-4ee3-97c4-a09b9cdf5625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:28.447818 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468c4df-cd02-4ee3-97c4-a09b9cdf5625 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:28.448071 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468c4df-cd02-4ee3-97c4-a09b9cdf5625 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:28.448433 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468c4df-cd02-4ee3-97c4-a09b9cdf5625 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:28.449289 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468c4df-cd02-4ee3-97c4-a09b9cdf5625 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:28.451133 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468c4df-cd02-4ee3-97c4-a09b9cdf5625 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:28.477465 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 863/1708] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:28 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:28.478704 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e4f57-6992-42c4-be58-ce10b12110e0 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:28.489887 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e4f57-6992-42c4-be58-ce10b12110e0 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:28.496315 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ca051-d23f-417e-8d96-fcf36f8a1df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:28.497052 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ca051-d23f-417e-8d96-fcf36f8a1df5 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:28.497335 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ca051-d23f-417e-8d96-fcf36f8a1df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:28.497607 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ca051-d23f-417e-8d96-fcf36f8a1df5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:28.498863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ca051-d23f-417e-8d96-fcf36f8a1df5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:28.501102 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ca051-d23f-417e-8d96-fcf36f8a1df5 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:28.511909 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 846/1709] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:28.517017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468c4df-cd02-4ee3-97c4-a09b9cdf5625 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:28.532256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468c4df-cd02-4ee3-97c4-a09b9cdf5625 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:28.532256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd2e9a-cb8c-42d6-a814-d56b926e134e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:28.532822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd2e9a-cb8c-42d6-a814-d56b926e134e admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:28.533101 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd2e9a-cb8c-42d6-a814-d56b926e134e admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:28.533374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd2e9a-cb8c-42d6-a814-d56b926e134e admin admin] PATH_INFO: `/v3/projects/aec90290c79d449bacd1fe65bb71105a/users/b75da2d334ed4e55aa5dc8540083fa95/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:28.550168 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd2e9a-cb8c-42d6-a814-d56b926e134e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aec90290c79d449bacd1fe65bb71105a, user_id=b75da2d334ed4e55aa5dc8540083fa95, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:28.552857 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd2e9a-cb8c-42d6-a814-d56b926e134e admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:28.569603 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 864/1710] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:28 2022] PUT /identity/v3/projects/aec90290c79d449bacd1fe65bb71105a/users/b75da2d334ed4e55aa5dc8540083fa95/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:28.574705 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ca051-d23f-417e-8d96-fcf36f8a1df5 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:28.588179 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ca051-d23f-417e-8d96-fcf36f8a1df5 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:28.652060 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004cb0d-015a-4a14-bff0-42a2c479606c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:28.652717 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004cb0d-015a-4a14-bff0-42a2c479606c admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:28.652979 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004cb0d-015a-4a14-bff0-42a2c479606c admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:28.653251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004cb0d-015a-4a14-bff0-42a2c479606c admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:28.653886 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004cb0d-015a-4a14-bff0-42a2c479606c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:28.656186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004cb0d-015a-4a14-bff0-42a2c479606c admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:28.657516 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:28.671713 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 847/1711] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:28.672700 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd2e9a-cb8c-42d6-a814-d56b926e134e admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:28.683430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd2e9a-cb8c-42d6-a814-d56b926e134e admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:28.690170 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08338f0-4931-4687-8dfd-97c59ec7692d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:28.691058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08338f0-4931-4687-8dfd-97c59ec7692d admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:28.691324 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08338f0-4931-4687-8dfd-97c59ec7692d admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:28.691569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08338f0-4931-4687-8dfd-97c59ec7692d admin admin] PATH_INFO: `/v3/projects/aec90290c79d449bacd1fe65bb71105a/users/b75da2d334ed4e55aa5dc8540083fa95/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:28.695363 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08338f0-4931-4687-8dfd-97c59ec7692d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aec90290c79d449bacd1fe65bb71105a, user_id=b75da2d334ed4e55aa5dc8540083fa95, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:28.696912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08338f0-4931-4687-8dfd-97c59ec7692d admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:28.710833 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 865/1712] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:28 2022] PUT /identity/v3/projects/aec90290c79d449bacd1fe65bb71105a/users/b75da2d334ed4e55aa5dc8540083fa95/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:28.723409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9377a1-7b69-4ff3-982c-f1bcf2649761 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:28.723511 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9377a1-7b69-4ff3-982c-f1bcf2649761 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:28.723744 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9377a1-7b69-4ff3-982c-f1bcf2649761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:28.752401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-ca9377a1-7b69-4ff3-982c-f1bcf2649761 None None] MFA Rules not processed for user `b75da2d334ed4e55aa5dc8540083fa95`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:28.753615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-ca9377a1-7b69-4ff3-982c-f1bcf2649761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:28.824840 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 848/1713] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:28 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:28.837399 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08338f0-4931-4687-8dfd-97c59ec7692d admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:28.840008 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e08338f0-4931-4687-8dfd-97c59ec7692d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:28.907665 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08338f0-4931-4687-8dfd-97c59ec7692d admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:28.912649 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3960a5a-05b8-478c-87cf-796c6c3183d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:28.913268 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3960a5a-05b8-478c-87cf-796c6c3183d6 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:28.913548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3960a5a-05b8-478c-87cf-796c6c3183d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:28.913824 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3960a5a-05b8-478c-87cf-796c6c3183d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:28.923785 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3960a5a-05b8-478c-87cf-796c6c3183d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:28.925271 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3960a5a-05b8-478c-87cf-796c6c3183d6 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:28.935210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 866/1714] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:28 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:29.475333 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9377a1-7b69-4ff3-982c-f1bcf2649761 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:29.486820 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9377a1-7b69-4ff3-982c-f1bcf2649761 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:29.496367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289c644-2110-4a1f-af46-ad9513aa0431 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:29.496982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289c644-2110-4a1f-af46-ad9513aa0431 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:29.497258 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289c644-2110-4a1f-af46-ad9513aa0431 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:29.497530 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289c644-2110-4a1f-af46-ad9513aa0431 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:29.555081 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1289c644-2110-4a1f-af46-ad9513aa0431 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:29.556565 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1289c644-2110-4a1f-af46-ad9513aa0431 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:29.566290 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 849/1715] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:32.153900 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3960a5a-05b8-478c-87cf-796c6c3183d6 service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:32.168053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-b3960a5a-05b8-478c-87cf-796c6c3183d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:32.221903 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:32.256937 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3960a5a-05b8-478c-87cf-796c6c3183d6 service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:32.262771 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566fc97-a333-4db5-9975-75412fe32c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:32.263549 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566fc97-a333-4db5-9975-75412fe32c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:32.263953 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566fc97-a333-4db5-9975-75412fe32c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:32.264275 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566fc97-a333-4db5-9975-75412fe32c72 admin admin] PATH_INFO: `/v3/users/fce5acf54e0047e6864df235eb560a0c` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:32.265869 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566fc97-a333-4db5-9975-75412fe32c72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fce5acf54e0047e6864df235eb560a0c)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:32.268385 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566fc97-a333-4db5-9975-75412fe32c72 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:32.346149 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-e566fc97-a333-4db5-9975-75412fe32c72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fce5acf54e0047e6864df235eb560a0c'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:32.347125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-e566fc97-a333-4db5-9975-75412fe32c72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fce5acf54e0047e6864df235eb560a0c'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:32.360990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-e566fc97-a333-4db5-9975-75412fe32c72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fce5acf54e0047e6864df235eb560a0c'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:32.369638 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-e566fc97-a333-4db5-9975-75412fe32c72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fce5acf54e0047e6864df235eb560a0c'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:32.377281 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-e566fc97-a333-4db5-9975-75412fe32c72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fce5acf54e0047e6864df235eb560a0c'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:32.383544 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 867/1716] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:32 2022] DELETE /identity/v3/users/fce5acf54e0047e6864df235eb560a0c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:32.388063 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8d28f-685e-4cf0-89b9-b326eec0319a None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:32.388547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8d28f-685e-4cf0-89b9-b326eec0319a None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:32.388796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8d28f-685e-4cf0-89b9-b326eec0319a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:32.409861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-d7e8d28f-685e-4cf0-89b9-b326eec0319a None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:32.411073 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d7e8d28f-685e-4cf0-89b9-b326eec0319a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:32.479750 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 850/1717] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:50:32.488985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566fc97-a333-4db5-9975-75412fe32c72 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:32.490944 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e566fc97-a333-4db5-9975-75412fe32c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:32.623237 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566fc97-a333-4db5-9975-75412fe32c72 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:32.647575 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a3443-12d6-4b02-ac8e-75b14f072ba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:32.649116 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a3443-12d6-4b02-ac8e-75b14f072ba1 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:32.649404 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a3443-12d6-4b02-ac8e-75b14f072ba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:32.649681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a3443-12d6-4b02-ac8e-75b14f072ba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:32.668210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a3443-12d6-4b02-ac8e-75b14f072ba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:32.671019 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a3443-12d6-4b02-ac8e-75b14f072ba1 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:32.692077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 868/1718] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:33.642505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8d28f-685e-4cf0-89b9-b326eec0319a None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:33.643997 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d7e8d28f-685e-4cf0-89b9-b326eec0319a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:33.658397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8d28f-685e-4cf0-89b9-b326eec0319a None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:33.687402 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f836d9-577b-496e-8ce4-ef4994c2a55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:33.687765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626318b-8075-4583-bc18-d4939b493acf None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:33.688326 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626318b-8075-4583-bc18-d4939b493acf None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:33.688326 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626318b-8075-4583-bc18-d4939b493acf None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:33.689905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f836d9-577b-496e-8ce4-ef4994c2a55c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:33.690215 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 869/1719] 10.222.0.36 () {54 vars in 921 bytes} [Sun Oct 30 00:50:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:50:33.690523 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f836d9-577b-496e-8ce4-ef4994c2a55c admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:33.690950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f836d9-577b-496e-8ce4-ef4994c2a55c admin admin] PATH_INFO: `/v3/projects/476eba79ed9e41d18d47762b95a27ebd` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:33.693388 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f836d9-577b-496e-8ce4-ef4994c2a55c admin admin] RBAC: Authorizing `identity:delete_project(project_id=476eba79ed9e41d18d47762b95a27ebd)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:33.695848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f836d9-577b-496e-8ce4-ef4994c2a55c admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:33.702479 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f834cac-5def-4261-978f-dbc4c5f18f88 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:33.702829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f834cac-5def-4261-978f-dbc4c5f18f88 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:33.703122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f834cac-5def-4261-978f-dbc4c5f18f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:33.718671 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:33.734984 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-4f834cac-5def-4261-978f-dbc4c5f18f88 None None] MFA Rules not processed for user `5ded89d48d6c4a15a5c5829f594f8776`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:33.736305 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4f834cac-5def-4261-978f-dbc4c5f18f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:33.763423 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b7f836d9-577b-496e-8ce4-ef4994c2a55c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '476eba79ed9e41d18d47762b95a27ebd', 'request_id': 'req-b7f836d9-577b-496e-8ce4-ef4994c2a55c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:33.778828 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b7f836d9-577b-496e-8ce4-ef4994c2a55c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '476eba79ed9e41d18d47762b95a27ebd', 'request_id': 'req-b7f836d9-577b-496e-8ce4-ef4994c2a55c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:33.791103 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b7f836d9-577b-496e-8ce4-ef4994c2a55c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '476eba79ed9e41d18d47762b95a27ebd', 'request_id': 'req-b7f836d9-577b-496e-8ce4-ef4994c2a55c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:33.809702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:33.817066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b7f836d9-577b-496e-8ce4-ef4994c2a55c admin admin] The token cache is being invalidate because project 476eba79ed9e41d18d47762b95a27ebd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:33.820250 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b7f836d9-577b-496e-8ce4-ef4994c2a55c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:33.821204 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b7f836d9-577b-496e-8ce4-ef4994c2a55c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:33.822970 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 851/1720] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:33 2022] DELETE /identity/v3/projects/476eba79ed9e41d18d47762b95a27ebd => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:33.827995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31038754-a820-40ab-8c57-523e763c674e None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:33.832525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31038754-a820-40ab-8c57-523e763c674e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:33.832812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31038754-a820-40ab-8c57-523e763c674e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:33.849718 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 870/1721] 10.222.0.36 () {58 vars in 1018 bytes} [Sun Oct 30 00:50:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:50:33.855030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-31038754-a820-40ab-8c57-523e763c674e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:33.856391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-31038754-a820-40ab-8c57-523e763c674e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:33.944019 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 852/1722] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:33.952787 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80827bd8-c998-46b0-8441-6a5cb935b22a None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:33.953257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80827bd8-c998-46b0-8441-6a5cb935b22a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:33.953660 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80827bd8-c998-46b0-8441-6a5cb935b22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:33.976317 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-80827bd8-c998-46b0-8441-6a5cb935b22a None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:33.977736 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-80827bd8-c998-46b0-8441-6a5cb935b22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:33.987296 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 871/1723] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:33.991583 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31038754-a820-40ab-8c57-523e763c674e None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:34.002391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31038754-a820-40ab-8c57-523e763c674e None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:34.007223 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3f80d-8ff4-4981-bda1-1c4c77b779bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:34.007916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3f80d-8ff4-4981-bda1-1c4c77b779bc admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.008296 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3f80d-8ff4-4981-bda1-1c4c77b779bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.008644 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3f80d-8ff4-4981-bda1-1c4c77b779bc admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.009371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3f80d-8ff4-4981-bda1-1c4c77b779bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:34.009746 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3f80d-8ff4-4981-bda1-1c4c77b779bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:34.011448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3f80d-8ff4-4981-bda1-1c4c77b779bc admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:34.025074 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 853/1724] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:34.029322 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80827bd8-c998-46b0-8441-6a5cb935b22a None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:34.038981 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3f80d-8ff4-4981-bda1-1c4c77b779bc admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:34.039554 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80827bd8-c998-46b0-8441-6a5cb935b22a None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:34.040705 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-46f3f80d-8ff4-4981-bda1-1c4c77b779bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:34.044846 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8cac2-2243-4792-8281-c0189f3d8e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:34.045766 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8cac2-2243-4792-8281-c0189f3d8e99 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.046000 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8cac2-2243-4792-8281-c0189f3d8e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.046280 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8cac2-2243-4792-8281-c0189f3d8e99 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.046938 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8cac2-2243-4792-8281-c0189f3d8e99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:34.048913 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8cac2-2243-4792-8281-c0189f3d8e99 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:34.057804 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3f80d-8ff4-4981-bda1-1c4c77b779bc admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:34.065692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59834c4d-fa27-43c3-88ed-017fb44b9d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:34.066339 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59834c4d-fa27-43c3-88ed-017fb44b9d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.066619 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59834c4d-fa27-43c3-88ed-017fb44b9d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.066894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59834c4d-fa27-43c3-88ed-017fb44b9d29 admin admin] PATH_INFO: `/v3/users/8135391c507d4f06aeb7e74160bbd038` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.068039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59834c4d-fa27-43c3-88ed-017fb44b9d29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8135391c507d4f06aeb7e74160bbd038)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:34.070013 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59834c4d-fa27-43c3-88ed-017fb44b9d29 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:34.088714 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 872/1725] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:34 2022] POST /identity/v3/projects => generated 399 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:34.089584 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8cac2-2243-4792-8281-c0189f3d8e99 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:34.092924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-71d8cac2-2243-4792-8281-c0189f3d8e99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:34.127647 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-59834c4d-fa27-43c3-88ed-017fb44b9d29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8135391c507d4f06aeb7e74160bbd038'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:34.128451 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-59834c4d-fa27-43c3-88ed-017fb44b9d29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8135391c507d4f06aeb7e74160bbd038'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:34.144027 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-59834c4d-fa27-43c3-88ed-017fb44b9d29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8135391c507d4f06aeb7e74160bbd038'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:34.151075 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-59834c4d-fa27-43c3-88ed-017fb44b9d29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8135391c507d4f06aeb7e74160bbd038'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:34.151812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-59834c4d-fa27-43c3-88ed-017fb44b9d29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8135391c507d4f06aeb7e74160bbd038'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:34.167921 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 854/1726] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:34 2022] DELETE /identity/v3/users/8135391c507d4f06aeb7e74160bbd038 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:34.169116 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59834c4d-fa27-43c3-88ed-017fb44b9d29 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:34.172735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-59834c4d-fa27-43c3-88ed-017fb44b9d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:34.174132 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8cac2-2243-4792-8281-c0189f3d8e99 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:34.178292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:34.267492 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b4272-dc2e-4a9d-be9b-df77371c30fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:34.268477 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b4272-dc2e-4a9d-be9b-df77371c30fc service placement] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.269293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b4272-dc2e-4a9d-be9b-df77371c30fc service placement] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.270012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b4272-dc2e-4a9d-be9b-df77371c30fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.272126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-195b4272-dc2e-4a9d-be9b-df77371c30fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:34.286575 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b4272-dc2e-4a9d-be9b-df77371c30fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:34.288282 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b4272-dc2e-4a9d-be9b-df77371c30fc service placement] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:34.291712 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59834c4d-fa27-43c3-88ed-017fb44b9d29 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:34.297874 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5d437-fa1c-4866-95ea-1f82ac3a9bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:34.298756 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5d437-fa1c-4866-95ea-1f82ac3a9bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.299117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5d437-fa1c-4866-95ea-1f82ac3a9bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.299467 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5d437-fa1c-4866-95ea-1f82ac3a9bd0 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.299891 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 873/1727] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:34.300211 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5d437-fa1c-4866-95ea-1f82ac3a9bd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:34.302200 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5d437-fa1c-4866-95ea-1f82ac3a9bd0 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:34.331775 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 855/1728] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:34 2022] POST /identity/v3/users => generated 300 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:34.335676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b4272-dc2e-4a9d-be9b-df77371c30fc service placement] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:34.347345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b4272-dc2e-4a9d-be9b-df77371c30fc service placement] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:34.352053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae969d-d1d4-404c-8eac-2bbf1d8f9832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:34.352779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae969d-d1d4-404c-8eac-2bbf1d8f9832 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.353092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae969d-d1d4-404c-8eac-2bbf1d8f9832 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.353424 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae969d-d1d4-404c-8eac-2bbf1d8f9832 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.354167 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae969d-d1d4-404c-8eac-2bbf1d8f9832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:34.355805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae969d-d1d4-404c-8eac-2bbf1d8f9832 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:34.371948 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 874/1729] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:34.373734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5d437-fa1c-4866-95ea-1f82ac3a9bd0 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:34.384418 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5d437-fa1c-4866-95ea-1f82ac3a9bd0 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:34.390814 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fff929-a83b-4b40-91dc-39457fa1b7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:34.391785 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fff929-a83b-4b40-91dc-39457fa1b7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.392317 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fff929-a83b-4b40-91dc-39457fa1b7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.392929 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fff929-a83b-4b40-91dc-39457fa1b7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.393511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae969d-d1d4-404c-8eac-2bbf1d8f9832 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:34.394007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fff929-a83b-4b40-91dc-39457fa1b7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:34.396197 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-52ae969d-d1d4-404c-8eac-2bbf1d8f9832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:34.396886 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fff929-a83b-4b40-91dc-39457fa1b7b9 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:34.416590 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae969d-d1d4-404c-8eac-2bbf1d8f9832 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:34.420123 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 856/1730] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:34.421895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fff929-a83b-4b40-91dc-39457fa1b7b9 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:34.422032 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f758b-181f-4023-b3f9-49f5375b591f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:34.422650 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f758b-181f-4023-b3f9-49f5375b591f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.422931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f758b-181f-4023-b3f9-49f5375b591f admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.423350 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f758b-181f-4023-b3f9-49f5375b591f admin admin] PATH_INFO: `/v3/projects/e1b5b4b97dfd468f923996968764d206` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.424561 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f758b-181f-4023-b3f9-49f5375b591f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1b5b4b97dfd468f923996968764d206)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:34.426478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f758b-181f-4023-b3f9-49f5375b591f admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:34.434443 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fff929-a83b-4b40-91dc-39457fa1b7b9 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:34.439361 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2161cd-ce6b-484e-af0b-9347bb7dd040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:34.440311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2161cd-ce6b-484e-af0b-9347bb7dd040 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.440597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2161cd-ce6b-484e-af0b-9347bb7dd040 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.440875 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2161cd-ce6b-484e-af0b-9347bb7dd040 admin admin] PATH_INFO: `/v3/projects/8e39fc04c0e44e9eb755ba05f3f989a1/users/25763b10a9b64a3a924f98655697390e/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.458119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2161cd-ce6b-484e-af0b-9347bb7dd040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e39fc04c0e44e9eb755ba05f3f989a1, user_id=25763b10a9b64a3a924f98655697390e, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:34.461154 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2161cd-ce6b-484e-af0b-9347bb7dd040 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:34.474000 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:34.477776 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 857/1731] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:34 2022] PUT /identity/v3/projects/8e39fc04c0e44e9eb755ba05f3f989a1/users/25763b10a9b64a3a924f98655697390e/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:34.481112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-260f758b-181f-4023-b3f9-49f5375b591f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1b5b4b97dfd468f923996968764d206', 'request_id': 'req-260f758b-181f-4023-b3f9-49f5375b591f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:34.489159 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77ff97-bfd1-4bad-8a75-5f342c35609b None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.489596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77ff97-bfd1-4bad-8a75-5f342c35609b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.490025 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77ff97-bfd1-4bad-8a75-5f342c35609b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.490413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-260f758b-181f-4023-b3f9-49f5375b591f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1b5b4b97dfd468f923996968764d206', 'request_id': 'req-260f758b-181f-4023-b3f9-49f5375b591f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:34.496649 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-260f758b-181f-4023-b3f9-49f5375b591f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1b5b4b97dfd468f923996968764d206', 'request_id': 'req-260f758b-181f-4023-b3f9-49f5375b591f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:34.508699 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-260f758b-181f-4023-b3f9-49f5375b591f admin admin] The token cache is being invalidate because project e1b5b4b97dfd468f923996968764d206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:34.508988 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-260f758b-181f-4023-b3f9-49f5375b591f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:34.509725 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-260f758b-181f-4023-b3f9-49f5375b591f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:34.512259 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 875/1732] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:34 2022] DELETE /identity/v3/projects/e1b5b4b97dfd468f923996968764d206 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:34.520113 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-9d77ff97-bfd1-4bad-8a75-5f342c35609b None None] MFA Rules not processed for user `25763b10a9b64a3a924f98655697390e`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:34.521353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-9d77ff97-bfd1-4bad-8a75-5f342c35609b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:34.523557 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56f130-1f40-4eb8-b612-12f8355a7fdb None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.523873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56f130-1f40-4eb8-b612-12f8355a7fdb None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.524168 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56f130-1f40-4eb8-b612-12f8355a7fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.543874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-0b56f130-1f40-4eb8-b612-12f8355a7fdb None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:34.545251 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-0b56f130-1f40-4eb8-b612-12f8355a7fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:34.606133 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 858/1733] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:34 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:50:34.618833 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77ff97-bfd1-4bad-8a75-5f342c35609b None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:34.620375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-9d77ff97-bfd1-4bad-8a75-5f342c35609b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:34.621697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 876/1734] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:50:34.630003 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad851018-109b-4a1e-baa7-c4d6b6d7e0a9 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.630288 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad851018-109b-4a1e-baa7-c4d6b6d7e0a9 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.630560 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad851018-109b-4a1e-baa7-c4d6b6d7e0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.650450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-ad851018-109b-4a1e-baa7-c4d6b6d7e0a9 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:34.651637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ad851018-109b-4a1e-baa7-c4d6b6d7e0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:34.660526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 877/1735] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:50:34.664770 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad851018-109b-4a1e-baa7-c4d6b6d7e0a9 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:34.674644 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:34.675147 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad851018-109b-4a1e-baa7-c4d6b6d7e0a9 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:34.680362 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f117a-67db-4d84-9dca-0ab05aac9f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:34.680544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77ff97-bfd1-4bad-8a75-5f342c35609b None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:34.680935 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f117a-67db-4d84-9dca-0ab05aac9f3e admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.681220 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f117a-67db-4d84-9dca-0ab05aac9f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.681485 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f117a-67db-4d84-9dca-0ab05aac9f3e admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.682129 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f117a-67db-4d84-9dca-0ab05aac9f3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:34.682424 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f117a-67db-4d84-9dca-0ab05aac9f3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:34.684012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f117a-67db-4d84-9dca-0ab05aac9f3e admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:34.685088 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81e70d-cc40-44fc-b04e-8918db5983e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:34.686216 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81e70d-cc40-44fc-b04e-8918db5983e2 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.686494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81e70d-cc40-44fc-b04e-8918db5983e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.686932 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81e70d-cc40-44fc-b04e-8918db5983e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.695761 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81e70d-cc40-44fc-b04e-8918db5983e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:34.697227 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81e70d-cc40-44fc-b04e-8918db5983e2 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:34.697359 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 878/1736] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:50:34.702688 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f117a-67db-4d84-9dca-0ab05aac9f3e admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:34.710181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 859/1737] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:34 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:50:34.712491 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f117a-67db-4d84-9dca-0ab05aac9f3e admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:34.717253 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb1a39-aec2-4977-8899-b3b94c599ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:34.717923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb1a39-aec2-4977-8899-b3b94c599ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.718195 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb1a39-aec2-4977-8899-b3b94c599ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.718462 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb1a39-aec2-4977-8899-b3b94c599ca9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.719082 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb1a39-aec2-4977-8899-b3b94c599ca9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:34.721001 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb1a39-aec2-4977-8899-b3b94c599ca9 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:34.752867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 879/1738] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:34 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:50:34.755228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81e70d-cc40-44fc-b04e-8918db5983e2 service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:34.766852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81e70d-cc40-44fc-b04e-8918db5983e2 service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:34.830692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876816c-6ba9-4efe-83d1-439a3e7ccc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:34.831514 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876816c-6ba9-4efe-83d1-439a3e7ccc5b admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.831627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876816c-6ba9-4efe-83d1-439a3e7ccc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.831851 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876816c-6ba9-4efe-83d1-439a3e7ccc5b admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.832528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876816c-6ba9-4efe-83d1-439a3e7ccc5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:34.834502 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876816c-6ba9-4efe-83d1-439a3e7ccc5b admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:34.863307 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 860/1739] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:34 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:34.867580 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb1a39-aec2-4977-8899-b3b94c599ca9 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:34.894573 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb1a39-aec2-4977-8899-b3b94c599ca9 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:34.902449 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9440b4-596a-46cd-b55c-8fe465ddae01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:34.903160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9440b4-596a-46cd-b55c-8fe465ddae01 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.903505 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9440b4-596a-46cd-b55c-8fe465ddae01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.903843 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9440b4-596a-46cd-b55c-8fe465ddae01 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.904865 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9440b4-596a-46cd-b55c-8fe465ddae01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:34.907116 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9440b4-596a-46cd-b55c-8fe465ddae01 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:34.917766 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 880/1740] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:34.923743 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876816c-6ba9-4efe-83d1-439a3e7ccc5b admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:34.934684 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876816c-6ba9-4efe-83d1-439a3e7ccc5b admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:34.940528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d028fa-8a7d-4ad8-8c76-faf60f6e0043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:34.941164 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d028fa-8a7d-4ad8-8c76-faf60f6e0043 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.941447 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d028fa-8a7d-4ad8-8c76-faf60f6e0043 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.941721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d028fa-8a7d-4ad8-8c76-faf60f6e0043 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.942365 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d028fa-8a7d-4ad8-8c76-faf60f6e0043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:34.944137 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d028fa-8a7d-4ad8-8c76-faf60f6e0043 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:34.954356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 861/1741] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:34.958614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9440b4-596a-46cd-b55c-8fe465ddae01 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:34.969893 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9440b4-596a-46cd-b55c-8fe465ddae01 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:34.976846 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3007c9b1-0b3a-4f09-bc8e-bbf92f96180b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:34.977713 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007c9b1-0b3a-4f09-bc8e-bbf92f96180b admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:34.978134 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007c9b1-0b3a-4f09-bc8e-bbf92f96180b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:34.978574 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007c9b1-0b3a-4f09-bc8e-bbf92f96180b admin admin] PATH_INFO: `/v3/projects/81ad74619f834ae49664c26013e93b30/users/416a49fce09443a3b48b82065c7e5964/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:34.997659 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3007c9b1-0b3a-4f09-bc8e-bbf92f96180b admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ad74619f834ae49664c26013e93b30, user_id=416a49fce09443a3b48b82065c7e5964, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:35.001560 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3007c9b1-0b3a-4f09-bc8e-bbf92f96180b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:35.025607 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 881/1742] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:34 2022] PUT /identity/v3/projects/81ad74619f834ae49664c26013e93b30/users/416a49fce09443a3b48b82065c7e5964/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:35.037698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e3711-7a97-4e56-9573-cda38ed68dc8 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:35.038066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e3711-7a97-4e56-9573-cda38ed68dc8 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:35.038390 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e3711-7a97-4e56-9573-cda38ed68dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:35.068306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-d50e3711-7a97-4e56-9573-cda38ed68dc8 None None] MFA Rules not processed for user `416a49fce09443a3b48b82065c7e5964`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:35.069529 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d50e3711-7a97-4e56-9573-cda38ed68dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:35.140118 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 862/1743] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:35.144796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3007c9b1-0b3a-4f09-bc8e-bbf92f96180b admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:35.233033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:35.245023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3007c9b1-0b3a-4f09-bc8e-bbf92f96180b admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:35.251333 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54110e30-c302-4645-9338-46ffa1303dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:35.252185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54110e30-c302-4645-9338-46ffa1303dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:35.252931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54110e30-c302-4645-9338-46ffa1303dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:35.253731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54110e30-c302-4645-9338-46ffa1303dc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:35.254570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54110e30-c302-4645-9338-46ffa1303dc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:35.256859 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54110e30-c302-4645-9338-46ffa1303dc6 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:35.293433 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 882/1744] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:35 2022] POST /identity/v3/projects => generated 381 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:35.298261 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e3711-7a97-4e56-9573-cda38ed68dc8 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:35.310137 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e3711-7a97-4e56-9573-cda38ed68dc8 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:35.367350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:35.375252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1fc8a-0d34-48ea-8ca4-d2cd8b6406c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:35.376496 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1fc8a-0d34-48ea-8ca4-d2cd8b6406c5 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:35.377016 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1fc8a-0d34-48ea-8ca4-d2cd8b6406c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:35.377595 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1fc8a-0d34-48ea-8ca4-d2cd8b6406c5 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:35.378223 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd1fc8a-0d34-48ea-8ca4-d2cd8b6406c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:35.380195 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd1fc8a-0d34-48ea-8ca4-d2cd8b6406c5 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:35.404152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 863/1745] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:35 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:35.406945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54110e30-c302-4645-9338-46ffa1303dc6 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:35.418229 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54110e30-c302-4645-9338-46ffa1303dc6 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:35.423128 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83563e-4faa-4e61-93a2-fcec3fdf81a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:35.423737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83563e-4faa-4e61-93a2-fcec3fdf81a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:35.424043 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83563e-4faa-4e61-93a2-fcec3fdf81a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:35.424327 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83563e-4faa-4e61-93a2-fcec3fdf81a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:35.424967 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83563e-4faa-4e61-93a2-fcec3fdf81a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:35.426566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83563e-4faa-4e61-93a2-fcec3fdf81a8 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:35.437633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 883/1746] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:35.442085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1fc8a-0d34-48ea-8ca4-d2cd8b6406c5 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:35.452976 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1fc8a-0d34-48ea-8ca4-d2cd8b6406c5 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:35.457591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff6b07-0972-405c-8dd6-f86ed433ddc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:35.458242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff6b07-0972-405c-8dd6-f86ed433ddc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:35.458518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff6b07-0972-405c-8dd6-f86ed433ddc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:35.458789 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff6b07-0972-405c-8dd6-f86ed433ddc4 admin admin] PATH_INFO: `/v3/projects/69e10bd25c1e480d9524b8633d6dcdb3/users/6a92321fb3cf4b3f94e892795facd5ba/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:35.474695 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff6b07-0972-405c-8dd6-f86ed433ddc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69e10bd25c1e480d9524b8633d6dcdb3, user_id=6a92321fb3cf4b3f94e892795facd5ba, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:35.477418 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff6b07-0972-405c-8dd6-f86ed433ddc4 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:35.489607 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 864/1747] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:35 2022] PUT /identity/v3/projects/69e10bd25c1e480d9524b8633d6dcdb3/users/6a92321fb3cf4b3f94e892795facd5ba/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:35.493707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83563e-4faa-4e61-93a2-fcec3fdf81a8 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:35.504387 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83563e-4faa-4e61-93a2-fcec3fdf81a8 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:35.561137 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1df7d5-8060-4953-b840-e68af8eae3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:35.561838 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1df7d5-8060-4953-b840-e68af8eae3df admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:35.562091 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1df7d5-8060-4953-b840-e68af8eae3df admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:35.562364 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1df7d5-8060-4953-b840-e68af8eae3df admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:35.563001 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1df7d5-8060-4953-b840-e68af8eae3df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:35.564649 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1df7d5-8060-4953-b840-e68af8eae3df admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:35.574557 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 884/1748] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:50:35.580063 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff6b07-0972-405c-8dd6-f86ed433ddc4 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:35.590408 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff6b07-0972-405c-8dd6-f86ed433ddc4 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:35.595055 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66828057-59d0-4e1e-8c81-4930fc3a6af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:35.595705 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66828057-59d0-4e1e-8c81-4930fc3a6af3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:35.595985 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66828057-59d0-4e1e-8c81-4930fc3a6af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:35.596288 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66828057-59d0-4e1e-8c81-4930fc3a6af3 admin admin] PATH_INFO: `/v3/projects/69e10bd25c1e480d9524b8633d6dcdb3/users/6a92321fb3cf4b3f94e892795facd5ba/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:35.598183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66828057-59d0-4e1e-8c81-4930fc3a6af3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69e10bd25c1e480d9524b8633d6dcdb3, user_id=6a92321fb3cf4b3f94e892795facd5ba, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:35.600961 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66828057-59d0-4e1e-8c81-4930fc3a6af3 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:35.611487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 865/1749] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:35 2022] PUT /identity/v3/projects/69e10bd25c1e480d9524b8633d6dcdb3/users/6a92321fb3cf4b3f94e892795facd5ba/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:35.621943 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f824a3-b60e-498f-81c9-62805d03e2e3 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:35.622231 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f824a3-b60e-498f-81c9-62805d03e2e3 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:35.622583 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f824a3-b60e-498f-81c9-62805d03e2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:35.652499 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-76f824a3-b60e-498f-81c9-62805d03e2e3 None None] MFA Rules not processed for user `6a92321fb3cf4b3f94e892795facd5ba`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:35.653706 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-76f824a3-b60e-498f-81c9-62805d03e2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:35.724843 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 885/1750] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:35.736889 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66828057-59d0-4e1e-8c81-4930fc3a6af3 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:35.738675 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-66828057-59d0-4e1e-8c81-4930fc3a6af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:35.796184 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66828057-59d0-4e1e-8c81-4930fc3a6af3 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:35.799794 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b960f8-4010-4f31-8953-32b8850d2dcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:35.800438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b960f8-4010-4f31-8953-32b8850d2dcc service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:35.800717 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b960f8-4010-4f31-8953-32b8850d2dcc service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:35.800988 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b960f8-4010-4f31-8953-32b8850d2dcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:35.809739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b960f8-4010-4f31-8953-32b8850d2dcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:35.812232 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b960f8-4010-4f31-8953-32b8850d2dcc service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:35.821780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 866/1751] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:35.903047 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f824a3-b60e-498f-81c9-62805d03e2e3 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:35.913138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f824a3-b60e-498f-81c9-62805d03e2e3 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:35.916757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb132c-0e79-430a-abb8-e3edc456ad50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:35.917359 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb132c-0e79-430a-abb8-e3edc456ad50 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:35.917641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb132c-0e79-430a-abb8-e3edc456ad50 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:35.917916 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb132c-0e79-430a-abb8-e3edc456ad50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:35.970688 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:35.984608 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb132c-0e79-430a-abb8-e3edc456ad50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:35.986184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb132c-0e79-430a-abb8-e3edc456ad50 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:35.995218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 886/1752] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:36.158347 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b960f8-4010-4f31-8953-32b8850d2dcc service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:36.169852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b960f8-4010-4f31-8953-32b8850d2dcc service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:36.174866 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d4c00-54a2-4ffa-b448-0b686ebb8f47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:36.175468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d4c00-54a2-4ffa-b448-0b686ebb8f47 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:36.175741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d4c00-54a2-4ffa-b448-0b686ebb8f47 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:36.176008 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d4c00-54a2-4ffa-b448-0b686ebb8f47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:36.177862 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-109d4c00-54a2-4ffa-b448-0b686ebb8f47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:36.239398 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d4c00-54a2-4ffa-b448-0b686ebb8f47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:36.240904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d4c00-54a2-4ffa-b448-0b686ebb8f47 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:36.250654 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 867/1753] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:38.377204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb132c-0e79-430a-abb8-e3edc456ad50 service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:38.378594 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a4cb132c-0e79-430a-abb8-e3edc456ad50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:38.447973 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb132c-0e79-430a-abb8-e3edc456ad50 service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:38.453345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea47b41-f8d3-481d-8d86-dd2d82596705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:38.453987 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea47b41-f8d3-481d-8d86-dd2d82596705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:38.454261 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea47b41-f8d3-481d-8d86-dd2d82596705 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:38.454532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea47b41-f8d3-481d-8d86-dd2d82596705 admin admin] PATH_INFO: `/v3/users/070faaa05a1a453f9bd6b0324fa168f1` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:38.455650 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea47b41-f8d3-481d-8d86-dd2d82596705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=070faaa05a1a453f9bd6b0324fa168f1)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:38.457655 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea47b41-f8d3-481d-8d86-dd2d82596705 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:38.505128 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7ea47b41-f8d3-481d-8d86-dd2d82596705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '070faaa05a1a453f9bd6b0324fa168f1'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:38.505698 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7ea47b41-f8d3-481d-8d86-dd2d82596705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '070faaa05a1a453f9bd6b0324fa168f1'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:38.518289 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7ea47b41-f8d3-481d-8d86-dd2d82596705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '070faaa05a1a453f9bd6b0324fa168f1'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:38.525026 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7ea47b41-f8d3-481d-8d86-dd2d82596705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '070faaa05a1a453f9bd6b0324fa168f1'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:38.532331 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7ea47b41-f8d3-481d-8d86-dd2d82596705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '070faaa05a1a453f9bd6b0324fa168f1'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:38.538136 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 887/1754] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:38 2022] DELETE /identity/v3/users/070faaa05a1a453f9bd6b0324fa168f1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:38.542551 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f83078-7c70-4549-8003-560c8f828e4c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:38.543012 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f83078-7c70-4549-8003-560c8f828e4c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:38.543127 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f83078-7c70-4549-8003-560c8f828e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:38.562911 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-17f83078-7c70-4549-8003-560c8f828e4c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:38.564162 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-17f83078-7c70-4549-8003-560c8f828e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:38.627032 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:38.643478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 868/1755] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:38.653479 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea47b41-f8d3-481d-8d86-dd2d82596705 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:38.654965 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-7ea47b41-f8d3-481d-8d86-dd2d82596705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:38.715746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea47b41-f8d3-481d-8d86-dd2d82596705 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:38.719620 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7723970-a3f7-4e3e-b77a-79ad3e595a1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:38.720264 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7723970-a3f7-4e3e-b77a-79ad3e595a1d service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:38.720545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7723970-a3f7-4e3e-b77a-79ad3e595a1d service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:38.720819 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7723970-a3f7-4e3e-b77a-79ad3e595a1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:38.729721 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7723970-a3f7-4e3e-b77a-79ad3e595a1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:38.731149 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7723970-a3f7-4e3e-b77a-79ad3e595a1d service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:38.740975 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 888/1756] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:39.022150 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f83078-7c70-4549-8003-560c8f828e4c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:39.023653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-17f83078-7c70-4549-8003-560c8f828e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:39.036178 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f83078-7c70-4549-8003-560c8f828e4c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:39.041163 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300c769-201d-4b09-afd1-dc0fb43bc40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:39.042257 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300c769-201d-4b09-afd1-dc0fb43bc40b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:39.042547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300c769-201d-4b09-afd1-dc0fb43bc40b admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:39.043154 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300c769-201d-4b09-afd1-dc0fb43bc40b admin admin] PATH_INFO: `/v3/projects/3a11b99fca8349fd868672bdc2d76193` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:39.044310 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300c769-201d-4b09-afd1-dc0fb43bc40b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a11b99fca8349fd868672bdc2d76193)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:39.047259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300c769-201d-4b09-afd1-dc0fb43bc40b admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:39.095638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-3300c769-201d-4b09-afd1-dc0fb43bc40b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a11b99fca8349fd868672bdc2d76193', 'request_id': 'req-3300c769-201d-4b09-afd1-dc0fb43bc40b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.102372 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-3300c769-201d-4b09-afd1-dc0fb43bc40b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a11b99fca8349fd868672bdc2d76193', 'request_id': 'req-3300c769-201d-4b09-afd1-dc0fb43bc40b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.109249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-3300c769-201d-4b09-afd1-dc0fb43bc40b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a11b99fca8349fd868672bdc2d76193', 'request_id': 'req-3300c769-201d-4b09-afd1-dc0fb43bc40b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.119565 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-3300c769-201d-4b09-afd1-dc0fb43bc40b admin admin] The token cache is being invalidate because project 3a11b99fca8349fd868672bdc2d76193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:39.119856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-3300c769-201d-4b09-afd1-dc0fb43bc40b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.120568 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-3300c769-201d-4b09-afd1-dc0fb43bc40b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.122101 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 869/1757] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:39 2022] DELETE /identity/v3/projects/3a11b99fca8349fd868672bdc2d76193 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:39.126415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7723970-a3f7-4e3e-b77a-79ad3e595a1d service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:39.135869 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7723970-a3f7-4e3e-b77a-79ad3e595a1d service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:39.140642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:39.194212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be7559-1aaa-4828-98b3-1d81aee57fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:39.194848 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be7559-1aaa-4828-98b3-1d81aee57fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:39.195125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be7559-1aaa-4828-98b3-1d81aee57fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:39.195396 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be7559-1aaa-4828-98b3-1d81aee57fc4 admin admin] PATH_INFO: `/v3/users/154d186c97fa4073b3e141765b0f6cb0` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:39.196525 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be7559-1aaa-4828-98b3-1d81aee57fc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=154d186c97fa4073b3e141765b0f6cb0)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:39.198458 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be7559-1aaa-4828-98b3-1d81aee57fc4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:39.246645 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-c9be7559-1aaa-4828-98b3-1d81aee57fc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '154d186c97fa4073b3e141765b0f6cb0'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.247302 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-c9be7559-1aaa-4828-98b3-1d81aee57fc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '154d186c97fa4073b3e141765b0f6cb0'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.260933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-c9be7559-1aaa-4828-98b3-1d81aee57fc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '154d186c97fa4073b3e141765b0f6cb0'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.268151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-c9be7559-1aaa-4828-98b3-1d81aee57fc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '154d186c97fa4073b3e141765b0f6cb0'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.277183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-c9be7559-1aaa-4828-98b3-1d81aee57fc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '154d186c97fa4073b3e141765b0f6cb0'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.284503 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 889/1758] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:39 2022] DELETE /identity/v3/users/154d186c97fa4073b3e141765b0f6cb0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:39.557228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300c769-201d-4b09-afd1-dc0fb43bc40b admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:39.558719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-3300c769-201d-4b09-afd1-dc0fb43bc40b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:39.572761 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300c769-201d-4b09-afd1-dc0fb43bc40b admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:39.634279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697a03e-f4b7-49e5-9178-5653af637906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:39.634919 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697a03e-f4b7-49e5-9178-5653af637906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:39.635201 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697a03e-f4b7-49e5-9178-5653af637906 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:39.635473 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697a03e-f4b7-49e5-9178-5653af637906 admin admin] PATH_INFO: `/v3/projects/278005942db84b23a399005c0d170389` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:39.636617 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f697a03e-f4b7-49e5-9178-5653af637906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=278005942db84b23a399005c0d170389)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:39.638525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f697a03e-f4b7-49e5-9178-5653af637906 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:39.692358 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f697a03e-f4b7-49e5-9178-5653af637906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '278005942db84b23a399005c0d170389', 'request_id': 'req-f697a03e-f4b7-49e5-9178-5653af637906'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.701669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f697a03e-f4b7-49e5-9178-5653af637906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '278005942db84b23a399005c0d170389', 'request_id': 'req-f697a03e-f4b7-49e5-9178-5653af637906'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.708206 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f697a03e-f4b7-49e5-9178-5653af637906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '278005942db84b23a399005c0d170389', 'request_id': 'req-f697a03e-f4b7-49e5-9178-5653af637906'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.718612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f697a03e-f4b7-49e5-9178-5653af637906 admin admin] The token cache is being invalidate because project 278005942db84b23a399005c0d170389 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:39.718902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f697a03e-f4b7-49e5-9178-5653af637906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.719569 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f697a03e-f4b7-49e5-9178-5653af637906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.721092 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 870/1759] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:39 2022] DELETE /identity/v3/projects/278005942db84b23a399005c0d170389 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:39.725254 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be7559-1aaa-4828-98b3-1d81aee57fc4 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:39.726635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c9be7559-1aaa-4828-98b3-1d81aee57fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:39.740212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be7559-1aaa-4828-98b3-1d81aee57fc4 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:39.798359 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ddc06f-5344-45e1-ad31-0b3bc13f9042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:39.798997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ddc06f-5344-45e1-ad31-0b3bc13f9042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:39.799278 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ddc06f-5344-45e1-ad31-0b3bc13f9042 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:39.799553 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ddc06f-5344-45e1-ad31-0b3bc13f9042 admin admin] PATH_INFO: `/v3/users/b75da2d334ed4e55aa5dc8540083fa95` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:39.800780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ddc06f-5344-45e1-ad31-0b3bc13f9042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b75da2d334ed4e55aa5dc8540083fa95)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:39.802648 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ddc06f-5344-45e1-ad31-0b3bc13f9042 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:39.836126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:39.851120 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-00ddc06f-5344-45e1-ad31-0b3bc13f9042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b75da2d334ed4e55aa5dc8540083fa95'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.851780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-00ddc06f-5344-45e1-ad31-0b3bc13f9042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b75da2d334ed4e55aa5dc8540083fa95'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.864491 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-00ddc06f-5344-45e1-ad31-0b3bc13f9042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b75da2d334ed4e55aa5dc8540083fa95'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.872012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-00ddc06f-5344-45e1-ad31-0b3bc13f9042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b75da2d334ed4e55aa5dc8540083fa95'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.879934 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-00ddc06f-5344-45e1-ad31-0b3bc13f9042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b75da2d334ed4e55aa5dc8540083fa95'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:39.886024 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 890/1760] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:39 2022] DELETE /identity/v3/users/b75da2d334ed4e55aa5dc8540083fa95 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:40.165799 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697a03e-f4b7-49e5-9178-5653af637906 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:40.167210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-f697a03e-f4b7-49e5-9178-5653af637906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:40.179444 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:40.183285 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697a03e-f4b7-49e5-9178-5653af637906 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:40.248007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da555fb-932d-44d8-b3bb-6ce06f75935b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:40.248904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da555fb-932d-44d8-b3bb-6ce06f75935b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:40.249551 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da555fb-932d-44d8-b3bb-6ce06f75935b admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:40.249551 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da555fb-932d-44d8-b3bb-6ce06f75935b admin admin] PATH_INFO: `/v3/projects/aec90290c79d449bacd1fe65bb71105a` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:40.250544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da555fb-932d-44d8-b3bb-6ce06f75935b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aec90290c79d449bacd1fe65bb71105a)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:40.252483 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da555fb-932d-44d8-b3bb-6ce06f75935b admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:40.298924 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-5da555fb-932d-44d8-b3bb-6ce06f75935b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aec90290c79d449bacd1fe65bb71105a', 'request_id': 'req-5da555fb-932d-44d8-b3bb-6ce06f75935b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:40.306720 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-5da555fb-932d-44d8-b3bb-6ce06f75935b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aec90290c79d449bacd1fe65bb71105a', 'request_id': 'req-5da555fb-932d-44d8-b3bb-6ce06f75935b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:40.313213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-5da555fb-932d-44d8-b3bb-6ce06f75935b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aec90290c79d449bacd1fe65bb71105a', 'request_id': 'req-5da555fb-932d-44d8-b3bb-6ce06f75935b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:40.323814 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-5da555fb-932d-44d8-b3bb-6ce06f75935b admin admin] The token cache is being invalidate because project aec90290c79d449bacd1fe65bb71105a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:40.324129 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-5da555fb-932d-44d8-b3bb-6ce06f75935b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:40.324776 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-5da555fb-932d-44d8-b3bb-6ce06f75935b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:40.327536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 871/1761] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:40 2022] DELETE /identity/v3/projects/aec90290c79d449bacd1fe65bb71105a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:40.333470 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee5e25-7091-4471-a700-d1b2fcb9b0f3 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:40.333756 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee5e25-7091-4471-a700-d1b2fcb9b0f3 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:40.334080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee5e25-7091-4471-a700-d1b2fcb9b0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:40.357312 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-d2ee5e25-7091-4471-a700-d1b2fcb9b0f3 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:40.358573 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d2ee5e25-7091-4471-a700-d1b2fcb9b0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:40.418328 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 891/1762] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:40.426682 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdde01-f2ee-45fa-9f93-60da7ff7a55d None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:40.426996 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdde01-f2ee-45fa-9f93-60da7ff7a55d None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:40.427270 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdde01-f2ee-45fa-9f93-60da7ff7a55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:40.447346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-b2bdde01-f2ee-45fa-9f93-60da7ff7a55d None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:40.448595 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b2bdde01-f2ee-45fa-9f93-60da7ff7a55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:40.457405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 872/1763] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:40.461409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee5e25-7091-4471-a700-d1b2fcb9b0f3 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:40.473539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee5e25-7091-4471-a700-d1b2fcb9b0f3 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:40.478136 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610a047-734c-48a9-b4a9-759b62cdb8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:40.478747 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610a047-734c-48a9-b4a9-759b62cdb8ca admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:40.479030 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610a047-734c-48a9-b4a9-759b62cdb8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:40.479307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610a047-734c-48a9-b4a9-759b62cdb8ca admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:40.479950 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610a047-734c-48a9-b4a9-759b62cdb8ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:40.480289 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610a047-734c-48a9-b4a9-759b62cdb8ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:40.481887 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610a047-734c-48a9-b4a9-759b62cdb8ca admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:40.493393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 892/1764] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:40.497641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdde01-f2ee-45fa-9f93-60da7ff7a55d None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:40.506958 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdde01-f2ee-45fa-9f93-60da7ff7a55d None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:40.511616 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f54014-752b-4c60-ade8-ea96728e4d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:40.512241 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f54014-752b-4c60-ade8-ea96728e4d38 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:40.512524 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f54014-752b-4c60-ade8-ea96728e4d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:40.512797 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f54014-752b-4c60-ade8-ea96728e4d38 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:40.513462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f54014-752b-4c60-ade8-ea96728e4d38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:40.516159 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f54014-752b-4c60-ade8-ea96728e4d38 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:40.543025 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 873/1765] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:40 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:40.547179 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610a047-734c-48a9-b4a9-759b62cdb8ca admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:40.556867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610a047-734c-48a9-b4a9-759b62cdb8ca admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:40.613658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb1599-9703-4c5a-8048-0acb5e9ddd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:40.614185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb1599-9703-4c5a-8048-0acb5e9ddd2a admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:40.614466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb1599-9703-4c5a-8048-0acb5e9ddd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:40.614743 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb1599-9703-4c5a-8048-0acb5e9ddd2a admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:40.615362 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb1599-9703-4c5a-8048-0acb5e9ddd2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:40.617294 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb1599-9703-4c5a-8048-0acb5e9ddd2a admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:40.638897 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 893/1766] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:40 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:40.642671 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f54014-752b-4c60-ade8-ea96728e4d38 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:40.653849 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f54014-752b-4c60-ade8-ea96728e4d38 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:40.659778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3a257-082f-4602-a628-f1cebbc337bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:40.660430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3a257-082f-4602-a628-f1cebbc337bb admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:40.660710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3a257-082f-4602-a628-f1cebbc337bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:40.660985 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3a257-082f-4602-a628-f1cebbc337bb admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:40.661650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3a257-082f-4602-a628-f1cebbc337bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:40.663241 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3a257-082f-4602-a628-f1cebbc337bb admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:40.677264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 874/1767] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:40.681239 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb1599-9703-4c5a-8048-0acb5e9ddd2a admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:40.691421 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb1599-9703-4c5a-8048-0acb5e9ddd2a admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:40.696067 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe21511-604a-41e5-a9b5-fc0673a6429a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:40.696698 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe21511-604a-41e5-a9b5-fc0673a6429a admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:40.696980 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe21511-604a-41e5-a9b5-fc0673a6429a admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:40.697248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe21511-604a-41e5-a9b5-fc0673a6429a admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:40.697880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe21511-604a-41e5-a9b5-fc0673a6429a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:40.699481 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe21511-604a-41e5-a9b5-fc0673a6429a admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:40.710725 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 894/1768] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:40.714904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3a257-082f-4602-a628-f1cebbc337bb admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:40.725804 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3a257-082f-4602-a628-f1cebbc337bb admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:40.730536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b7051-d367-4cf5-b1e8-1c1b93be47bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:40.731184 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b7051-d367-4cf5-b1e8-1c1b93be47bd admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:40.731470 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b7051-d367-4cf5-b1e8-1c1b93be47bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:40.731749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b7051-d367-4cf5-b1e8-1c1b93be47bd admin admin] PATH_INFO: `/v3/projects/006d909321a04694aba1ce7cb28dd3e3/users/76b36595a0a6411ba30d0f48eec1fff9/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:40.746958 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b7051-d367-4cf5-b1e8-1c1b93be47bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=006d909321a04694aba1ce7cb28dd3e3, user_id=76b36595a0a6411ba30d0f48eec1fff9, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:40.749988 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b7051-d367-4cf5-b1e8-1c1b93be47bd admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:40.762859 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 875/1769] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:40 2022] PUT /identity/v3/projects/006d909321a04694aba1ce7cb28dd3e3/users/76b36595a0a6411ba30d0f48eec1fff9/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:40.774420 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6823a-9e3d-421e-b1dc-501937bad806 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:40.774912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6823a-9e3d-421e-b1dc-501937bad806 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:40.775004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6823a-9e3d-421e-b1dc-501937bad806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:40.803628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:40.806902 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-39c6823a-9e3d-421e-b1dc-501937bad806 None None] MFA Rules not processed for user `76b36595a0a6411ba30d0f48eec1fff9`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:40.809044 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-39c6823a-9e3d-421e-b1dc-501937bad806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:40.877953 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 895/1770] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:50:40.882630 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b7051-d367-4cf5-b1e8-1c1b93be47bd admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:40.915986 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:40.947360 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b7051-d367-4cf5-b1e8-1c1b93be47bd admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:40.952788 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18d81e-a2a0-4e62-bb0b-abcb2d9d80be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:40.953393 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18d81e-a2a0-4e62-bb0b-abcb2d9d80be admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:40.953677 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18d81e-a2a0-4e62-bb0b-abcb2d9d80be admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:40.953953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18d81e-a2a0-4e62-bb0b-abcb2d9d80be admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:40.954699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18d81e-a2a0-4e62-bb0b-abcb2d9d80be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:40.956657 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18d81e-a2a0-4e62-bb0b-abcb2d9d80be admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:40.983729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 876/1771] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:40 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:40.988116 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6823a-9e3d-421e-b1dc-501937bad806 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:40.998675 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6823a-9e3d-421e-b1dc-501937bad806 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:41.054343 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb2b85-69ec-4d31-b83c-65cf6554a763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:41.054955 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb2b85-69ec-4d31-b83c-65cf6554a763 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:41.055242 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb2b85-69ec-4d31-b83c-65cf6554a763 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:41.055517 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb2b85-69ec-4d31-b83c-65cf6554a763 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:41.056187 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb2b85-69ec-4d31-b83c-65cf6554a763 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:41.058111 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb2b85-69ec-4d31-b83c-65cf6554a763 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:41.080165 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 896/1772] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:40 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:41.084350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18d81e-a2a0-4e62-bb0b-abcb2d9d80be admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:41.095216 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18d81e-a2a0-4e62-bb0b-abcb2d9d80be admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:41.099770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6480f-91d6-40f3-838f-84d3ee7e9dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:41.100399 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6480f-91d6-40f3-838f-84d3ee7e9dab admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:41.100682 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6480f-91d6-40f3-838f-84d3ee7e9dab admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:41.100951 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6480f-91d6-40f3-838f-84d3ee7e9dab admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:41.101578 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6480f-91d6-40f3-838f-84d3ee7e9dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:41.103166 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6480f-91d6-40f3-838f-84d3ee7e9dab admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:41.113641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 877/1773] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:41.117500 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb2b85-69ec-4d31-b83c-65cf6554a763 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:41.127759 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb2b85-69ec-4d31-b83c-65cf6554a763 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:41.132501 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add3e0c-278a-4bcc-9c68-bf2dbc545a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:41.133112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add3e0c-278a-4bcc-9c68-bf2dbc545a50 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:41.133389 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add3e0c-278a-4bcc-9c68-bf2dbc545a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:41.133677 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add3e0c-278a-4bcc-9c68-bf2dbc545a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:41.134336 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add3e0c-278a-4bcc-9c68-bf2dbc545a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:41.135924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add3e0c-278a-4bcc-9c68-bf2dbc545a50 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:41.146494 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 897/1774] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:41.150538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6480f-91d6-40f3-838f-84d3ee7e9dab admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:41.161408 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6480f-91d6-40f3-838f-84d3ee7e9dab admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:41.166041 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15eddb8-35fc-436c-89bc-476252e6e5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:41.166699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15eddb8-35fc-436c-89bc-476252e6e5af admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:41.166982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15eddb8-35fc-436c-89bc-476252e6e5af admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:41.167274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15eddb8-35fc-436c-89bc-476252e6e5af admin admin] PATH_INFO: `/v3/projects/f1db477a7cb74cd4a5e489e52e7f9f67/users/ad1d05ba02334fbda52913138b00038f/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:41.182195 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15eddb8-35fc-436c-89bc-476252e6e5af admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1db477a7cb74cd4a5e489e52e7f9f67, user_id=ad1d05ba02334fbda52913138b00038f, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:41.185071 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15eddb8-35fc-436c-89bc-476252e6e5af admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:41.200576 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 878/1775] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:41 2022] PUT /identity/v3/projects/f1db477a7cb74cd4a5e489e52e7f9f67/users/ad1d05ba02334fbda52913138b00038f/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:41.212824 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbbec9-0d56-4890-8acd-68c29b475fa1 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:41.213109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbbec9-0d56-4890-8acd-68c29b475fa1 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:41.213384 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbbec9-0d56-4890-8acd-68c29b475fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:41.243407 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-1dbbbec9-0d56-4890-8acd-68c29b475fa1 None None] MFA Rules not processed for user `ad1d05ba02334fbda52913138b00038f`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:41.244814 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-1dbbbec9-0d56-4890-8acd-68c29b475fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:41.318645 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 898/1776] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:41.323033 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15eddb8-35fc-436c-89bc-476252e6e5af admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:41.387526 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15eddb8-35fc-436c-89bc-476252e6e5af admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:41.392472 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbc615-8aec-4cac-b59a-ccf2b7f55bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:41.393084 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbc615-8aec-4cac-b59a-ccf2b7f55bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:41.393366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbc615-8aec-4cac-b59a-ccf2b7f55bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:41.393642 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbc615-8aec-4cac-b59a-ccf2b7f55bb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:41.394273 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbc615-8aec-4cac-b59a-ccf2b7f55bb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:41.396261 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbc615-8aec-4cac-b59a-ccf2b7f55bb7 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:41.423448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 879/1777] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:41 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:41.427609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbbec9-0d56-4890-8acd-68c29b475fa1 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:41.438329 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbbec9-0d56-4890-8acd-68c29b475fa1 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:41.446315 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:41.498802 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ecc7b-a3f8-4a98-a3e5-da954d73ce15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:41.499418 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ecc7b-a3f8-4a98-a3e5-da954d73ce15 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:41.499698 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ecc7b-a3f8-4a98-a3e5-da954d73ce15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:41.499972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ecc7b-a3f8-4a98-a3e5-da954d73ce15 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:41.500635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ecc7b-a3f8-4a98-a3e5-da954d73ce15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:41.502553 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ecc7b-a3f8-4a98-a3e5-da954d73ce15 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:41.523217 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 899/1778] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:41 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:41.527115 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbc615-8aec-4cac-b59a-ccf2b7f55bb7 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:41.538231 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbc615-8aec-4cac-b59a-ccf2b7f55bb7 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:41.542729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f6294-592f-447b-873f-ac934b645f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:41.543320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f6294-592f-447b-873f-ac934b645f1c admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:41.543598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f6294-592f-447b-873f-ac934b645f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:41.543869 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f6294-592f-447b-873f-ac934b645f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:41.544525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f6294-592f-447b-873f-ac934b645f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:41.547337 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f6294-592f-447b-873f-ac934b645f1c admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:41.558772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 880/1779] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:41.562980 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ecc7b-a3f8-4a98-a3e5-da954d73ce15 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:41.572940 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ecc7b-a3f8-4a98-a3e5-da954d73ce15 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:41.577924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5362cef-e38e-4163-8918-4b2d17b5ae7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:41.578558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5362cef-e38e-4163-8918-4b2d17b5ae7d admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:41.578833 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5362cef-e38e-4163-8918-4b2d17b5ae7d admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:41.579101 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5362cef-e38e-4163-8918-4b2d17b5ae7d admin admin] PATH_INFO: `/v3/projects/e58fd3008c0c412495ac9ddf6decb5bf/users/c92c1a09719846a8a9abd367993941d0/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:41.594957 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5362cef-e38e-4163-8918-4b2d17b5ae7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e58fd3008c0c412495ac9ddf6decb5bf, user_id=c92c1a09719846a8a9abd367993941d0, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:41.597631 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5362cef-e38e-4163-8918-4b2d17b5ae7d admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:41.610443 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 900/1780] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:41 2022] PUT /identity/v3/projects/e58fd3008c0c412495ac9ddf6decb5bf/users/c92c1a09719846a8a9abd367993941d0/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:41.614359 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f6294-592f-447b-873f-ac934b645f1c admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:41.624907 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f6294-592f-447b-873f-ac934b645f1c admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:41.674648 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:41.683300 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7424a3-9b47-4f11-8d39-233cc423e113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:41.684139 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7424a3-9b47-4f11-8d39-233cc423e113 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:41.684655 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7424a3-9b47-4f11-8d39-233cc423e113 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:41.684965 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7424a3-9b47-4f11-8d39-233cc423e113 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:41.686206 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7424a3-9b47-4f11-8d39-233cc423e113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:41.688279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7424a3-9b47-4f11-8d39-233cc423e113 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:41.699680 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 881/1781] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:41.704100 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5362cef-e38e-4163-8918-4b2d17b5ae7d admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:41.714509 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5362cef-e38e-4163-8918-4b2d17b5ae7d admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:41.719351 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b096f1-bf0f-4f4d-bcc5-3aec15a96f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:41.720010 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b096f1-bf0f-4f4d-bcc5-3aec15a96f43 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:41.720317 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b096f1-bf0f-4f4d-bcc5-3aec15a96f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:41.720590 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b096f1-bf0f-4f4d-bcc5-3aec15a96f43 admin admin] PATH_INFO: `/v3/projects/e58fd3008c0c412495ac9ddf6decb5bf/users/c92c1a09719846a8a9abd367993941d0/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:41.722569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b096f1-bf0f-4f4d-bcc5-3aec15a96f43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e58fd3008c0c412495ac9ddf6decb5bf, user_id=c92c1a09719846a8a9abd367993941d0, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:41.725230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b096f1-bf0f-4f4d-bcc5-3aec15a96f43 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:41.736927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 901/1782] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:41 2022] PUT /identity/v3/projects/e58fd3008c0c412495ac9ddf6decb5bf/users/c92c1a09719846a8a9abd367993941d0/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:41.746805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47692510-db75-4f0d-9c21-a732b82d420d None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:41.747152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47692510-db75-4f0d-9c21-a732b82d420d None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:41.747492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47692510-db75-4f0d-9c21-a732b82d420d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:41.776402 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-47692510-db75-4f0d-9c21-a732b82d420d None None] MFA Rules not processed for user `c92c1a09719846a8a9abd367993941d0`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:41.777667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-47692510-db75-4f0d-9c21-a732b82d420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:41.845987 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 882/1783] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:41.858091 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b096f1-bf0f-4f4d-bcc5-3aec15a96f43 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:41.860134 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-04b096f1-bf0f-4f4d-bcc5-3aec15a96f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:41.924353 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b096f1-bf0f-4f4d-bcc5-3aec15a96f43 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:41.927978 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730cf7a-d4f7-4b92-a241-1802723ac216 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:41.928558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730cf7a-d4f7-4b92-a241-1802723ac216 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:41.928839 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730cf7a-d4f7-4b92-a241-1802723ac216 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:41.929111 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730cf7a-d4f7-4b92-a241-1802723ac216 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:41.938873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730cf7a-d4f7-4b92-a241-1802723ac216 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:41.940595 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730cf7a-d4f7-4b92-a241-1802723ac216 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:41.949823 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 902/1784] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:50:41.992443 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47692510-db75-4f0d-9c21-a732b82d420d None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:41.993912 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-47692510-db75-4f0d-9c21-a732b82d420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:42.006914 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47692510-db75-4f0d-9c21-a732b82d420d None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:42.010569 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf10a9-4902-4eda-a338-c980e425d089 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:42.011170 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf10a9-4902-4eda-a338-c980e425d089 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:42.011458 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf10a9-4902-4eda-a338-c980e425d089 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:42.011731 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf10a9-4902-4eda-a338-c980e425d089 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:42.068955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf10a9-4902-4eda-a338-c980e425d089 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:42.070407 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf10a9-4902-4eda-a338-c980e425d089 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:42.079303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 883/1785] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:51.615319 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730cf7a-d4f7-4b92-a241-1802723ac216 service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:51.616891 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4730cf7a-d4f7-4b92-a241-1802723ac216 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:51.683208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730cf7a-d4f7-4b92-a241-1802723ac216 service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:51.688551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf0e56-b391-4e14-9721-ce0e010ba334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:51.689177 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf0e56-b391-4e14-9721-ce0e010ba334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:51.689454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf0e56-b391-4e14-9721-ce0e010ba334 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:51.690700 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf0e56-b391-4e14-9721-ce0e010ba334 admin admin] PATH_INFO: `/v3/users/d7fd3fb1e7484c179a6f57d3cc6be741` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:51.691127 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf0e56-b391-4e14-9721-ce0e010ba334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7fd3fb1e7484c179a6f57d3cc6be741)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:51.693109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf0e56-b391-4e14-9721-ce0e010ba334 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:51.699714 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:51.744014 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-9fdf0e56-b391-4e14-9721-ce0e010ba334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7fd3fb1e7484c179a6f57d3cc6be741'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:51.744689 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-9fdf0e56-b391-4e14-9721-ce0e010ba334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7fd3fb1e7484c179a6f57d3cc6be741'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:51.757465 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-9fdf0e56-b391-4e14-9721-ce0e010ba334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7fd3fb1e7484c179a6f57d3cc6be741'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:51.764139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-9fdf0e56-b391-4e14-9721-ce0e010ba334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7fd3fb1e7484c179a6f57d3cc6be741'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:51.771800 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-9fdf0e56-b391-4e14-9721-ce0e010ba334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7fd3fb1e7484c179a6f57d3cc6be741'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:51.777567 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 903/1786] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:51 2022] DELETE /identity/v3/users/d7fd3fb1e7484c179a6f57d3cc6be741 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:51.783181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1bbf5-446e-40c7-8bb9-922f08d8f190 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:51.783708 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1bbf5-446e-40c7-8bb9-922f08d8f190 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:51.783850 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1bbf5-446e-40c7-8bb9-922f08d8f190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:51.804703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-68c1bbf5-446e-40c7-8bb9-922f08d8f190 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:51.805908 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-68c1bbf5-446e-40c7-8bb9-922f08d8f190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:51.867948 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 884/1787] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:51.878267 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf0e56-b391-4e14-9721-ce0e010ba334 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:51.879684 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-9fdf0e56-b391-4e14-9721-ce0e010ba334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:51.943641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf0e56-b391-4e14-9721-ce0e010ba334 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:51.954884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd689e47-cfe3-4882-9127-272b414e2e4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:51.955638 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd689e47-cfe3-4882-9127-272b414e2e4f service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:51.955926 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd689e47-cfe3-4882-9127-272b414e2e4f service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:51.956243 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd689e47-cfe3-4882-9127-272b414e2e4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:51.966234 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd689e47-cfe3-4882-9127-272b414e2e4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:51.968503 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd689e47-cfe3-4882-9127-272b414e2e4f service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:51.978492 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 904/1788] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:52.274525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1bbf5-446e-40c7-8bb9-922f08d8f190 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:52.275991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-68c1bbf5-446e-40c7-8bb9-922f08d8f190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:52.300744 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1bbf5-446e-40c7-8bb9-922f08d8f190 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:52.306832 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec21472b-9098-438a-8749-e62556170915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:52.307449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21472b-9098-438a-8749-e62556170915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:52.307733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21472b-9098-438a-8749-e62556170915 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:52.308029 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21472b-9098-438a-8749-e62556170915 admin admin] PATH_INFO: `/v3/projects/66bd01489a0a43dd84e5ac0d0a80aa2d` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:52.309165 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec21472b-9098-438a-8749-e62556170915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66bd01489a0a43dd84e5ac0d0a80aa2d)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:52.312650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec21472b-9098-438a-8749-e62556170915 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:52.326165 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:52.393822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd689e47-cfe3-4882-9127-272b414e2e4f service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:52.395663 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-fd689e47-cfe3-4882-9127-272b414e2e4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:52.404683 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-ec21472b-9098-438a-8749-e62556170915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66bd01489a0a43dd84e5ac0d0a80aa2d', 'request_id': 'req-ec21472b-9098-438a-8749-e62556170915'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:52.409273 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd689e47-cfe3-4882-9127-272b414e2e4f service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:52.419049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-ec21472b-9098-438a-8749-e62556170915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66bd01489a0a43dd84e5ac0d0a80aa2d', 'request_id': 'req-ec21472b-9098-438a-8749-e62556170915'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:52.427983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-ec21472b-9098-438a-8749-e62556170915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66bd01489a0a43dd84e5ac0d0a80aa2d', 'request_id': 'req-ec21472b-9098-438a-8749-e62556170915'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:52.444685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-ec21472b-9098-438a-8749-e62556170915 admin admin] The token cache is being invalidate because project 66bd01489a0a43dd84e5ac0d0a80aa2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:52.445082 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-ec21472b-9098-438a-8749-e62556170915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:52.445854 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-ec21472b-9098-438a-8749-e62556170915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:52.451104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 885/1789] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:52 2022] DELETE /identity/v3/projects/66bd01489a0a43dd84e5ac0d0a80aa2d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:52.451104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec21472b-9098-438a-8749-e62556170915 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:52.453032 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-ec21472b-9098-438a-8749-e62556170915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:52.485118 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27b700-c72c-457d-8c21-962c48e1ca69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:52.486200 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27b700-c72c-457d-8c21-962c48e1ca69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:52.486583 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27b700-c72c-457d-8c21-962c48e1ca69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:52.487035 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27b700-c72c-457d-8c21-962c48e1ca69 admin admin] PATH_INFO: `/v3/users/25763b10a9b64a3a924f98655697390e` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:52.488654 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27b700-c72c-457d-8c21-962c48e1ca69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25763b10a9b64a3a924f98655697390e)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:52.491037 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27b700-c72c-457d-8c21-962c48e1ca69 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:52.530780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:52.535343 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec21472b-9098-438a-8749-e62556170915 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:52.540724 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37a6b2-6c68-4618-bcc9-31cd217666ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:52.541368 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37a6b2-6c68-4618-bcc9-31cd217666ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:52.541653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37a6b2-6c68-4618-bcc9-31cd217666ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:52.541925 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37a6b2-6c68-4618-bcc9-31cd217666ce admin admin] PATH_INFO: `/v3/users/76b36595a0a6411ba30d0f48eec1fff9` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:52.543073 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37a6b2-6c68-4618-bcc9-31cd217666ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=76b36595a0a6411ba30d0f48eec1fff9)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:52.545070 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37a6b2-6c68-4618-bcc9-31cd217666ce admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:52.546600 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7d27b700-c72c-457d-8c21-962c48e1ca69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25763b10a9b64a3a924f98655697390e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:52.547714 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7d27b700-c72c-457d-8c21-962c48e1ca69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25763b10a9b64a3a924f98655697390e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:52.561672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7d27b700-c72c-457d-8c21-962c48e1ca69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25763b10a9b64a3a924f98655697390e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:52.569617 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7d27b700-c72c-457d-8c21-962c48e1ca69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25763b10a9b64a3a924f98655697390e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:52.576761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7d27b700-c72c-457d-8c21-962c48e1ca69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25763b10a9b64a3a924f98655697390e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:52.583444 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 905/1790] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:52 2022] DELETE /identity/v3/users/25763b10a9b64a3a924f98655697390e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:52.584733 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3b31a-10e6-4fe3-952b-d7feed42aff9 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:52.585011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3b31a-10e6-4fe3-952b-d7feed42aff9 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:52.585289 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3b31a-10e6-4fe3-952b-d7feed42aff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:52.598014 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2b37a6b2-6c68-4618-bcc9-31cd217666ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76b36595a0a6411ba30d0f48eec1fff9'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:52.598769 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2b37a6b2-6c68-4618-bcc9-31cd217666ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76b36595a0a6411ba30d0f48eec1fff9'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:52.606899 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-38e3b31a-10e6-4fe3-952b-d7feed42aff9 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:52.609149 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-38e3b31a-10e6-4fe3-952b-d7feed42aff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:52.611552 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2b37a6b2-6c68-4618-bcc9-31cd217666ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76b36595a0a6411ba30d0f48eec1fff9'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:52.619148 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2b37a6b2-6c68-4618-bcc9-31cd217666ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76b36595a0a6411ba30d0f48eec1fff9'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:52.620095 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2b37a6b2-6c68-4618-bcc9-31cd217666ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76b36595a0a6411ba30d0f48eec1fff9'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:52.634886 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 886/1791] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:52 2022] DELETE /identity/v3/users/76b36595a0a6411ba30d0f48eec1fff9 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:52.636339 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d9e99-07de-4280-bfeb-046712269447 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:52.636624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d9e99-07de-4280-bfeb-046712269447 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:52.636898 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d9e99-07de-4280-bfeb-046712269447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:52.656435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-d97d9e99-07de-4280-bfeb-046712269447 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:52.657650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d97d9e99-07de-4280-bfeb-046712269447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:52.671432 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:52.681390 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 906/1792] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:52.681390 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408933b-c12b-4aa4-8e30-f3bd26956d7f None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:52.681390 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408933b-c12b-4aa4-8e30-f3bd26956d7f None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:52.681390 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408933b-c12b-4aa4-8e30-f3bd26956d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:52.699742 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-5408933b-c12b-4aa4-8e30-f3bd26956d7f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:52.700978 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-5408933b-c12b-4aa4-8e30-f3bd26956d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:52.723954 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 887/1793] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:52.728408 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e80bd6-47d0-419b-9de9-30c6ca928523 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:52.728695 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e80bd6-47d0-419b-9de9-30c6ca928523 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:52.728974 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e80bd6-47d0-419b-9de9-30c6ca928523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:52.750205 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-a0e80bd6-47d0-419b-9de9-30c6ca928523 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:52.751406 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a0e80bd6-47d0-419b-9de9-30c6ca928523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:52.759331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 888/1794] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:52.760211 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e80bd6-47d0-419b-9de9-30c6ca928523 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:52.761432 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a0e80bd6-47d0-419b-9de9-30c6ca928523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:52.762268 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 907/1795] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:52.766286 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408933b-c12b-4aa4-8e30-f3bd26956d7f None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:52.776599 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408933b-c12b-4aa4-8e30-f3bd26956d7f None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:52.781946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e912c0-2c18-4d8d-8642-3cd233963ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:52.782580 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e912c0-2c18-4d8d-8642-3cd233963ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:52.782860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e912c0-2c18-4d8d-8642-3cd233963ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:52.783159 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e912c0-2c18-4d8d-8642-3cd233963ff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:52.783804 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e912c0-2c18-4d8d-8642-3cd233963ff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:52.784162 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e912c0-2c18-4d8d-8642-3cd233963ff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:52.785739 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e912c0-2c18-4d8d-8642-3cd233963ff7 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:52.800118 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 908/1796] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:52.800118 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e912c0-2c18-4d8d-8642-3cd233963ff7 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:52.800659 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d5e912c0-2c18-4d8d-8642-3cd233963ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:52.822949 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e80bd6-47d0-419b-9de9-30c6ca928523 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:52.826620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cab4b-cb98-44ec-9f88-324f123d5a67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:52.827260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cab4b-cb98-44ec-9f88-324f123d5a67 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:52.827566 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cab4b-cb98-44ec-9f88-324f123d5a67 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:52.827821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cab4b-cb98-44ec-9f88-324f123d5a67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:52.836806 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232cab4b-cb98-44ec-9f88-324f123d5a67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:52.838232 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232cab4b-cb98-44ec-9f88-324f123d5a67 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:52.841649 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:52.849233 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 889/1797] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:52.850270 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cab4b-cb98-44ec-9f88-324f123d5a67 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:52.860926 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cab4b-cb98-44ec-9f88-324f123d5a67 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:52.862797 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e912c0-2c18-4d8d-8642-3cd233963ff7 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:52.866013 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93028ce-effe-4d78-bc80-64160a21d2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:52.866621 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93028ce-effe-4d78-bc80-64160a21d2b4 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:52.866903 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93028ce-effe-4d78-bc80-64160a21d2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:52.867010 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472a8fa-abda-42ce-90e8-bed81da284c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:52.867183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93028ce-effe-4d78-bc80-64160a21d2b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:52.867511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472a8fa-abda-42ce-90e8-bed81da284c4 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:52.867787 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472a8fa-abda-42ce-90e8-bed81da284c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:52.867893 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93028ce-effe-4d78-bc80-64160a21d2b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:52.868061 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472a8fa-abda-42ce-90e8-bed81da284c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:52.869851 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93028ce-effe-4d78-bc80-64160a21d2b4 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:52.885574 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472a8fa-abda-42ce-90e8-bed81da284c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:52.887563 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472a8fa-abda-42ce-90e8-bed81da284c4 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:52.902454 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 890/1798] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:52 2022] POST /identity/v3/projects => generated 359 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:52.907121 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93028ce-effe-4d78-bc80-64160a21d2b4 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:52.907314 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 909/1799] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:52.976165 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93028ce-effe-4d78-bc80-64160a21d2b4 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:52.982115 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0979ffd-510e-4f51-bc07-274b36d45274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:52.982115 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0979ffd-510e-4f51-bc07-274b36d45274 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:52.982115 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0979ffd-510e-4f51-bc07-274b36d45274 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:52.982432 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0979ffd-510e-4f51-bc07-274b36d45274 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:52.983571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0979ffd-510e-4f51-bc07-274b36d45274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:52.985895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0979ffd-510e-4f51-bc07-274b36d45274 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:53.012355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 891/1800] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:52 2022] POST /identity/v3/users => generated 280 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:50:53.017257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472a8fa-abda-42ce-90e8-bed81da284c4 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:53.032356 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472a8fa-abda-42ce-90e8-bed81da284c4 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:53.046511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3b8cb-93c7-403e-8a5a-d5be8d8eac18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:53.047443 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3b8cb-93c7-403e-8a5a-d5be8d8eac18 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:53.047740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3b8cb-93c7-403e-8a5a-d5be8d8eac18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:53.048003 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3b8cb-93c7-403e-8a5a-d5be8d8eac18 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:53.048701 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3b8cb-93c7-403e-8a5a-d5be8d8eac18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:53.050332 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3b8cb-93c7-403e-8a5a-d5be8d8eac18 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:53.061959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 910/1801] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:50:53.066729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0979ffd-510e-4f51-bc07-274b36d45274 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:53.078652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0979ffd-510e-4f51-bc07-274b36d45274 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:53.082867 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555edef-3bd8-40e1-95d6-d8500747f729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:53.083569 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555edef-3bd8-40e1-95d6-d8500747f729 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:53.084046 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555edef-3bd8-40e1-95d6-d8500747f729 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:53.084427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555edef-3bd8-40e1-95d6-d8500747f729 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:53.085232 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555edef-3bd8-40e1-95d6-d8500747f729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:53.086953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555edef-3bd8-40e1-95d6-d8500747f729 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:53.100117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 892/1802] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:53.102304 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3b8cb-93c7-403e-8a5a-d5be8d8eac18 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:53.114694 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3b8cb-93c7-403e-8a5a-d5be8d8eac18 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:53.119509 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357354a-7991-4ae2-9671-bd58bcd10aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:53.120227 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357354a-7991-4ae2-9671-bd58bcd10aa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:53.120511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357354a-7991-4ae2-9671-bd58bcd10aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:53.120782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357354a-7991-4ae2-9671-bd58bcd10aa2 admin admin] PATH_INFO: `/v3/projects/cb54ce13fa734785af74ad48f5451437/users/93d68ca6fb894995a42bb131ff825359/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:53.140367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357354a-7991-4ae2-9671-bd58bcd10aa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb54ce13fa734785af74ad48f5451437, user_id=93d68ca6fb894995a42bb131ff825359, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:53.143462 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357354a-7991-4ae2-9671-bd58bcd10aa2 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:53.159998 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 911/1803] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:53 2022] PUT /identity/v3/projects/cb54ce13fa734785af74ad48f5451437/users/93d68ca6fb894995a42bb131ff825359/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:53.173578 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3d17c-94e2-4ba1-b26f-9c505e017a35 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:53.173880 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3d17c-94e2-4ba1-b26f-9c505e017a35 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:53.174165 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3d17c-94e2-4ba1-b26f-9c505e017a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:53.206685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:53.217250 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-4eb3d17c-94e2-4ba1-b26f-9c505e017a35 None None] MFA Rules not processed for user `93d68ca6fb894995a42bb131ff825359`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:53.221309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4eb3d17c-94e2-4ba1-b26f-9c505e017a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:53.303138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 893/1804] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:53 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:53.334827 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357354a-7991-4ae2-9671-bd58bcd10aa2 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:53.336605 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f357354a-7991-4ae2-9671-bd58bcd10aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:53.347141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3d17c-94e2-4ba1-b26f-9c505e017a35 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:53.350437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4eb3d17c-94e2-4ba1-b26f-9c505e017a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:53.444601 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357354a-7991-4ae2-9671-bd58bcd10aa2 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:53.449447 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a8186-2f68-463d-92af-a673cac2187e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:53.450631 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a8186-2f68-463d-92af-a673cac2187e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:53.450998 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a8186-2f68-463d-92af-a673cac2187e admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:53.451286 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a8186-2f68-463d-92af-a673cac2187e admin admin] PATH_INFO: `/v3/projects/8e39fc04c0e44e9eb755ba05f3f989a1` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:53.452609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3d17c-94e2-4ba1-b26f-9c505e017a35 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:53.452733 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a8186-2f68-463d-92af-a673cac2187e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e39fc04c0e44e9eb755ba05f3f989a1)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:53.454855 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a8186-2f68-463d-92af-a673cac2187e admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:53.457666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0bcaf-1bfa-48ae-9936-1fab412c5781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:53.458675 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0bcaf-1bfa-48ae-9936-1fab412c5781 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:53.459030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0bcaf-1bfa-48ae-9936-1fab412c5781 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:53.459362 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0bcaf-1bfa-48ae-9936-1fab412c5781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:53.471359 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0bcaf-1bfa-48ae-9936-1fab412c5781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:53.474140 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0bcaf-1bfa-48ae-9936-1fab412c5781 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:53.485785 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 894/1805] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:53 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:50:53.487137 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0bcaf-1bfa-48ae-9936-1fab412c5781 service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:53.488771 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-68b0bcaf-1bfa-48ae-9936-1fab412c5781 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:53.507849 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0bcaf-1bfa-48ae-9936-1fab412c5781 service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:53.521149 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-6a0a8186-2f68-463d-92af-a673cac2187e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e39fc04c0e44e9eb755ba05f3f989a1', 'request_id': 'req-6a0a8186-2f68-463d-92af-a673cac2187e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.533307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-6a0a8186-2f68-463d-92af-a673cac2187e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e39fc04c0e44e9eb755ba05f3f989a1', 'request_id': 'req-6a0a8186-2f68-463d-92af-a673cac2187e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.541792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-6a0a8186-2f68-463d-92af-a673cac2187e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e39fc04c0e44e9eb755ba05f3f989a1', 'request_id': 'req-6a0a8186-2f68-463d-92af-a673cac2187e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.556971 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-6a0a8186-2f68-463d-92af-a673cac2187e admin admin] The token cache is being invalidate because project 8e39fc04c0e44e9eb755ba05f3f989a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:53.557266 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-6a0a8186-2f68-463d-92af-a673cac2187e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.557948 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-6a0a8186-2f68-463d-92af-a673cac2187e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.560208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 912/1806] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:53 2022] DELETE /identity/v3/projects/8e39fc04c0e44e9eb755ba05f3f989a1 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:53.561852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a8186-2f68-463d-92af-a673cac2187e admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:53.564689 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-6a0a8186-2f68-463d-92af-a673cac2187e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:53.575730 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ac510-a4a4-49a9-9260-4a9f5f04f611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:53.576442 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ac510-a4a4-49a9-9260-4a9f5f04f611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:53.577233 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ac510-a4a4-49a9-9260-4a9f5f04f611 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:53.577233 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ac510-a4a4-49a9-9260-4a9f5f04f611 admin admin] PATH_INFO: `/v3/projects/006d909321a04694aba1ce7cb28dd3e3` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:53.578507 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ac510-a4a4-49a9-9260-4a9f5f04f611 admin admin] RBAC: Authorizing `identity:delete_project(project_id=006d909321a04694aba1ce7cb28dd3e3)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:53.581532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ac510-a4a4-49a9-9260-4a9f5f04f611 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:53.624850 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:53.649717 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f19ac510-a4a4-49a9-9260-4a9f5f04f611 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '006d909321a04694aba1ce7cb28dd3e3', 'request_id': 'req-f19ac510-a4a4-49a9-9260-4a9f5f04f611'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.653248 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:53.661703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f19ac510-a4a4-49a9-9260-4a9f5f04f611 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '006d909321a04694aba1ce7cb28dd3e3', 'request_id': 'req-f19ac510-a4a4-49a9-9260-4a9f5f04f611'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.665472 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a8186-2f68-463d-92af-a673cac2187e admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:53.671610 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f19ac510-a4a4-49a9-9260-4a9f5f04f611 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '006d909321a04694aba1ce7cb28dd3e3', 'request_id': 'req-f19ac510-a4a4-49a9-9260-4a9f5f04f611'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.684379 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f19ac510-a4a4-49a9-9260-4a9f5f04f611 admin admin] The token cache is being invalidate because project 006d909321a04694aba1ce7cb28dd3e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:53.684699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f19ac510-a4a4-49a9-9260-4a9f5f04f611 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.685401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f19ac510-a4a4-49a9-9260-4a9f5f04f611 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.687493 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 895/1807] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:53 2022] DELETE /identity/v3/projects/006d909321a04694aba1ce7cb28dd3e3 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:53.688635 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ac510-a4a4-49a9-9260-4a9f5f04f611 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:53.690592 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-f19ac510-a4a4-49a9-9260-4a9f5f04f611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:53.746624 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca1517-29f7-4870-94e8-ceaf754911fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:53.747390 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca1517-29f7-4870-94e8-ceaf754911fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:53.747603 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca1517-29f7-4870-94e8-ceaf754911fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:53.747876 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca1517-29f7-4870-94e8-ceaf754911fc admin admin] PATH_INFO: `/v3/users/93d68ca6fb894995a42bb131ff825359` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:53.749182 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca1517-29f7-4870-94e8-ceaf754911fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=93d68ca6fb894995a42bb131ff825359)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:53.752221 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca1517-29f7-4870-94e8-ceaf754911fc admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:53.766564 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ac510-a4a4-49a9-9260-4a9f5f04f611 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:53.771283 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae74f4-fc38-4c42-9a64-b261537f53da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:53.771925 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae74f4-fc38-4c42-9a64-b261537f53da admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:53.772231 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae74f4-fc38-4c42-9a64-b261537f53da admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:53.772509 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae74f4-fc38-4c42-9a64-b261537f53da admin admin] PATH_INFO: `/v3/users/416a49fce09443a3b48b82065c7e5964` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:53.773789 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae74f4-fc38-4c42-9a64-b261537f53da admin admin] RBAC: Authorizing `identity:delete_user(user_id=416a49fce09443a3b48b82065c7e5964)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:53.775571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae74f4-fc38-4c42-9a64-b261537f53da admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:53.804796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d3ca1517-29f7-4870-94e8-ceaf754911fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93d68ca6fb894995a42bb131ff825359'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.805552 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d3ca1517-29f7-4870-94e8-ceaf754911fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93d68ca6fb894995a42bb131ff825359'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.818799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d3ca1517-29f7-4870-94e8-ceaf754911fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93d68ca6fb894995a42bb131ff825359'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.826376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d3ca1517-29f7-4870-94e8-ceaf754911fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93d68ca6fb894995a42bb131ff825359'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.828923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-34ae74f4-fc38-4c42-9a64-b261537f53da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '416a49fce09443a3b48b82065c7e5964'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.829600 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-34ae74f4-fc38-4c42-9a64-b261537f53da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '416a49fce09443a3b48b82065c7e5964'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.835551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d3ca1517-29f7-4870-94e8-ceaf754911fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93d68ca6fb894995a42bb131ff825359'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.841943 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 913/1808] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:53 2022] DELETE /identity/v3/users/93d68ca6fb894995a42bb131ff825359 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:53.842375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-34ae74f4-fc38-4c42-9a64-b261537f53da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '416a49fce09443a3b48b82065c7e5964'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.843362 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca1517-29f7-4870-94e8-ceaf754911fc admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:53.844767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d3ca1517-29f7-4870-94e8-ceaf754911fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:53.850308 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-34ae74f4-fc38-4c42-9a64-b261537f53da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '416a49fce09443a3b48b82065c7e5964'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.851509 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-34ae74f4-fc38-4c42-9a64-b261537f53da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '416a49fce09443a3b48b82065c7e5964'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:53.866367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 896/1809] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:53 2022] DELETE /identity/v3/users/416a49fce09443a3b48b82065c7e5964 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:53.869370 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48383a4-ce01-4a20-bc6f-9eb03375aa83 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:53.869370 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48383a4-ce01-4a20-bc6f-9eb03375aa83 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:53.869370 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48383a4-ce01-4a20-bc6f-9eb03375aa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:53.890290 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-e48383a4-ce01-4a20-bc6f-9eb03375aa83 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:53.891966 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e48383a4-ce01-4a20-bc6f-9eb03375aa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:53.917026 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca1517-29f7-4870-94e8-ceaf754911fc admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:53.922665 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:53.944795 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:53.959818 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 897/1810] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:53.961146 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67240fd-396e-40b5-977d-8ced9c582f5c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:53.961429 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67240fd-396e-40b5-977d-8ced9c582f5c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:53.961700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67240fd-396e-40b5-977d-8ced9c582f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:53.980762 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-c67240fd-396e-40b5-977d-8ced9c582f5c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:53.982113 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c67240fd-396e-40b5-977d-8ced9c582f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:53.983789 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd10e39-c6d2-49d3-a029-99711822a664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:53.984616 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd10e39-c6d2-49d3-a029-99711822a664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:53.984920 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd10e39-c6d2-49d3-a029-99711822a664 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:53.985232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd10e39-c6d2-49d3-a029-99711822a664 admin admin] PATH_INFO: `/v3/users/ad1d05ba02334fbda52913138b00038f` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:53.986367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd10e39-c6d2-49d3-a029-99711822a664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad1d05ba02334fbda52913138b00038f)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:53.988448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd10e39-c6d2-49d3-a029-99711822a664 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:53.992313 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 898/1811] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:53.996220 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67240fd-396e-40b5-977d-8ced9c582f5c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:53.997536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c67240fd-396e-40b5-977d-8ced9c582f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:54.036453 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-fbd10e39-c6d2-49d3-a029-99711822a664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad1d05ba02334fbda52913138b00038f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.037279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-fbd10e39-c6d2-49d3-a029-99711822a664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad1d05ba02334fbda52913138b00038f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.050298 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-fbd10e39-c6d2-49d3-a029-99711822a664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad1d05ba02334fbda52913138b00038f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.060950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67240fd-396e-40b5-977d-8ced9c582f5c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:54.063270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-fbd10e39-c6d2-49d3-a029-99711822a664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad1d05ba02334fbda52913138b00038f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.066863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a09587-4b59-42a5-bbff-6cdad5619041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:54.067803 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a09587-4b59-42a5-bbff-6cdad5619041 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:54.068100 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a09587-4b59-42a5-bbff-6cdad5619041 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:54.068488 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a09587-4b59-42a5-bbff-6cdad5619041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:54.070170 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-82a09587-4b59-42a5-bbff-6cdad5619041 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:54.072454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-fbd10e39-c6d2-49d3-a029-99711822a664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad1d05ba02334fbda52913138b00038f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.079350 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 914/1812] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:53 2022] DELETE /identity/v3/users/ad1d05ba02334fbda52913138b00038f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:54.079999 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd10e39-c6d2-49d3-a029-99711822a664 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:54.081401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-fbd10e39-c6d2-49d3-a029-99711822a664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:54.085478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a09587-4b59-42a5-bbff-6cdad5619041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:54.086940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a09587-4b59-42a5-bbff-6cdad5619041 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:54.098488 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 899/1813] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:54.173271 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd10e39-c6d2-49d3-a029-99711822a664 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:54.177092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c0347-abe8-4e1b-867d-4703e2187b87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:54.177712 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c0347-abe8-4e1b-867d-4703e2187b87 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:54.177992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c0347-abe8-4e1b-867d-4703e2187b87 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:54.178266 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c0347-abe8-4e1b-867d-4703e2187b87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:54.180071 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-211c0347-abe8-4e1b-867d-4703e2187b87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:54.230980 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:54.275895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c0347-abe8-4e1b-867d-4703e2187b87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:54.279128 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c0347-abe8-4e1b-867d-4703e2187b87 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:54.305952 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 915/1814] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:54.615869 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a09587-4b59-42a5-bbff-6cdad5619041 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:54.617745 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-82a09587-4b59-42a5-bbff-6cdad5619041 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:54.627275 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c0347-abe8-4e1b-867d-4703e2187b87 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:54.629232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-211c0347-abe8-4e1b-867d-4703e2187b87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:54.631312 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a09587-4b59-42a5-bbff-6cdad5619041 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:54.636209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96c5cf-5130-4132-bc23-ebbfe8745de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:54.637723 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96c5cf-5130-4132-bc23-ebbfe8745de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:54.638013 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96c5cf-5130-4132-bc23-ebbfe8745de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:54.638293 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96c5cf-5130-4132-bc23-ebbfe8745de4 admin admin] PATH_INFO: `/v3/projects/cb54ce13fa734785af74ad48f5451437` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:54.640649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96c5cf-5130-4132-bc23-ebbfe8745de4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb54ce13fa734785af74ad48f5451437)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:54.642599 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96c5cf-5130-4132-bc23-ebbfe8745de4 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:54.647855 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c0347-abe8-4e1b-867d-4703e2187b87 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:54.658769 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd070c-2d29-45e8-84df-a350058b804b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:54.660001 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd070c-2d29-45e8-84df-a350058b804b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:54.660557 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd070c-2d29-45e8-84df-a350058b804b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:54.661055 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd070c-2d29-45e8-84df-a350058b804b admin admin] PATH_INFO: `/v3/projects/f1db477a7cb74cd4a5e489e52e7f9f67` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:54.671272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd070c-2d29-45e8-84df-a350058b804b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1db477a7cb74cd4a5e489e52e7f9f67)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:54.674554 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd070c-2d29-45e8-84df-a350058b804b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:54.719944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-ca96c5cf-5130-4132-bc23-ebbfe8745de4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb54ce13fa734785af74ad48f5451437', 'request_id': 'req-ca96c5cf-5130-4132-bc23-ebbfe8745de4'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.730523 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-ca96c5cf-5130-4132-bc23-ebbfe8745de4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb54ce13fa734785af74ad48f5451437', 'request_id': 'req-ca96c5cf-5130-4132-bc23-ebbfe8745de4'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.757428 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-ca96c5cf-5130-4132-bc23-ebbfe8745de4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb54ce13fa734785af74ad48f5451437', 'request_id': 'req-ca96c5cf-5130-4132-bc23-ebbfe8745de4'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.759029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7ffd070c-2d29-45e8-84df-a350058b804b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1db477a7cb74cd4a5e489e52e7f9f67', 'request_id': 'req-7ffd070c-2d29-45e8-84df-a350058b804b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.769698 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7ffd070c-2d29-45e8-84df-a350058b804b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1db477a7cb74cd4a5e489e52e7f9f67', 'request_id': 'req-7ffd070c-2d29-45e8-84df-a350058b804b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.771693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-ca96c5cf-5130-4132-bc23-ebbfe8745de4 admin admin] The token cache is being invalidate because project cb54ce13fa734785af74ad48f5451437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:54.772012 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-ca96c5cf-5130-4132-bc23-ebbfe8745de4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.772710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-ca96c5cf-5130-4132-bc23-ebbfe8745de4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.774231 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 900/1815] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:54 2022] DELETE /identity/v3/projects/cb54ce13fa734785af74ad48f5451437 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:54.775734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7ffd070c-2d29-45e8-84df-a350058b804b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1db477a7cb74cd4a5e489e52e7f9f67', 'request_id': 'req-7ffd070c-2d29-45e8-84df-a350058b804b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.777031 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96c5cf-5130-4132-bc23-ebbfe8745de4 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:54.779178 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-ca96c5cf-5130-4132-bc23-ebbfe8745de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:54.787748 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7ffd070c-2d29-45e8-84df-a350058b804b admin admin] The token cache is being invalidate because project f1db477a7cb74cd4a5e489e52e7f9f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:54.788039 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7ffd070c-2d29-45e8-84df-a350058b804b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.788797 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7ffd070c-2d29-45e8-84df-a350058b804b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.790403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 916/1816] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:54 2022] DELETE /identity/v3/projects/f1db477a7cb74cd4a5e489e52e7f9f67 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:54.791709 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc3fbc-dbfe-4b12-9c09-a93b83e55d40 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:54.791995 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc3fbc-dbfe-4b12-9c09-a93b83e55d40 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:54.792302 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc3fbc-dbfe-4b12-9c09-a93b83e55d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:54.813169 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-7fbc3fbc-dbfe-4b12-9c09-a93b83e55d40 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:54.814389 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-7fbc3fbc-dbfe-4b12-9c09-a93b83e55d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:54.856407 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96c5cf-5130-4132-bc23-ebbfe8745de4 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:54.876917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:54.884672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 917/1817] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:54.885910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc3fbc-dbfe-4b12-9c09-a93b83e55d40 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:54.887208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-7fbc3fbc-dbfe-4b12-9c09-a93b83e55d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:54.902426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc3fbc-dbfe-4b12-9c09-a93b83e55d40 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:54.907850 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a2bb0-7fe9-4ddb-82ad-27a61ac67cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:54.908644 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a2bb0-7fe9-4ddb-82ad-27a61ac67cda admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:54.908927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a2bb0-7fe9-4ddb-82ad-27a61ac67cda admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:54.909204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a2bb0-7fe9-4ddb-82ad-27a61ac67cda admin admin] PATH_INFO: `/v3/users/c92c1a09719846a8a9abd367993941d0` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:54.910352 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a2bb0-7fe9-4ddb-82ad-27a61ac67cda admin admin] RBAC: Authorizing `identity:delete_user(user_id=c92c1a09719846a8a9abd367993941d0)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:54.912323 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a2bb0-7fe9-4ddb-82ad-27a61ac67cda admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:54.929378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745804f-de9e-4f12-a39a-22c399e479ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:54.930021 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745804f-de9e-4f12-a39a-22c399e479ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:54.930306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745804f-de9e-4f12-a39a-22c399e479ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:54.930582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745804f-de9e-4f12-a39a-22c399e479ca admin admin] PATH_INFO: `/v3/projects/81ad74619f834ae49664c26013e93b30` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:54.931693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745804f-de9e-4f12-a39a-22c399e479ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=81ad74619f834ae49664c26013e93b30)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:54.933777 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745804f-de9e-4f12-a39a-22c399e479ca admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:54.961984 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-210a2bb0-7fe9-4ddb-82ad-27a61ac67cda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c92c1a09719846a8a9abd367993941d0'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.962594 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-210a2bb0-7fe9-4ddb-82ad-27a61ac67cda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c92c1a09719846a8a9abd367993941d0'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.968822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:54.977514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-210a2bb0-7fe9-4ddb-82ad-27a61ac67cda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c92c1a09719846a8a9abd367993941d0'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.985157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-9745804f-de9e-4f12-a39a-22c399e479ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81ad74619f834ae49664c26013e93b30', 'request_id': 'req-9745804f-de9e-4f12-a39a-22c399e479ca'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.985477 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-210a2bb0-7fe9-4ddb-82ad-27a61ac67cda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c92c1a09719846a8a9abd367993941d0'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.992750 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-9745804f-de9e-4f12-a39a-22c399e479ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81ad74619f834ae49664c26013e93b30', 'request_id': 'req-9745804f-de9e-4f12-a39a-22c399e479ca'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.994570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-210a2bb0-7fe9-4ddb-82ad-27a61ac67cda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c92c1a09719846a8a9abd367993941d0'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:54.999984 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-9745804f-de9e-4f12-a39a-22c399e479ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81ad74619f834ae49664c26013e93b30', 'request_id': 'req-9745804f-de9e-4f12-a39a-22c399e479ca'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:55.001450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 918/1818] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:54 2022] DELETE /identity/v3/users/c92c1a09719846a8a9abd367993941d0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:55.002755 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c8ad0-7ffc-405e-973a-9eb0dec34912 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:55.002958 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c8ad0-7ffc-405e-973a-9eb0dec34912 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:55.003232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c8ad0-7ffc-405e-973a-9eb0dec34912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:55.012717 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-9745804f-de9e-4f12-a39a-22c399e479ca admin admin] The token cache is being invalidate because project 81ad74619f834ae49664c26013e93b30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:55.013012 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-9745804f-de9e-4f12-a39a-22c399e479ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:55.013701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-9745804f-de9e-4f12-a39a-22c399e479ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:55.015963 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 901/1819] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:54 2022] DELETE /identity/v3/projects/81ad74619f834ae49664c26013e93b30 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:55.020276 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745804f-de9e-4f12-a39a-22c399e479ca admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:55.021647 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-9745804f-de9e-4f12-a39a-22c399e479ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:55.023445 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-089c8ad0-7ffc-405e-973a-9eb0dec34912 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:55.024653 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-089c8ad0-7ffc-405e-973a-9eb0dec34912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:55.098255 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745804f-de9e-4f12-a39a-22c399e479ca admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:55.102133 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 919/1820] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:55.103034 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c8ad0-7ffc-405e-973a-9eb0dec34912 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:55.104000 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037dfbc9-38d1-45ee-aad1-c805c11c151e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:55.104678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dfbc9-38d1-45ee-aad1-c805c11c151e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:55.104963 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dfbc9-38d1-45ee-aad1-c805c11c151e admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:55.105240 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dfbc9-38d1-45ee-aad1-c805c11c151e admin admin] PATH_INFO: `/v3/users/6a92321fb3cf4b3f94e892795facd5ba` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:55.106445 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037dfbc9-38d1-45ee-aad1-c805c11c151e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a92321fb3cf4b3f94e892795facd5ba)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:55.108476 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037dfbc9-38d1-45ee-aad1-c805c11c151e admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:55.114618 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c8ad0-7ffc-405e-973a-9eb0dec34912 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:55.119828 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f3809-e6ce-488d-8e2c-824880959c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:55.120491 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f3809-e6ce-488d-8e2c-824880959c7c admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:55.120777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f3809-e6ce-488d-8e2c-824880959c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:55.121051 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f3809-e6ce-488d-8e2c-824880959c7c admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:55.121704 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f3809-e6ce-488d-8e2c-824880959c7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:55.122172 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f3809-e6ce-488d-8e2c-824880959c7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:55.124493 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f3809-e6ce-488d-8e2c-824880959c7c admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:55.144341 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 920/1821] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:55.148786 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f3809-e6ce-488d-8e2c-824880959c7c admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:55.160628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f3809-e6ce-488d-8e2c-824880959c7c admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:55.167654 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787321d4-2411-493d-8c73-96daf0a14a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:55.168317 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787321d4-2411-493d-8c73-96daf0a14a3b admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:55.168601 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787321d4-2411-493d-8c73-96daf0a14a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:55.172967 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787321d4-2411-493d-8c73-96daf0a14a3b admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:55.173649 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787321d4-2411-493d-8c73-96daf0a14a3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:55.175626 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787321d4-2411-493d-8c73-96daf0a14a3b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:55.194110 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-037dfbc9-38d1-45ee-aad1-c805c11c151e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a92321fb3cf4b3f94e892795facd5ba'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:55.195353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-037dfbc9-38d1-45ee-aad1-c805c11c151e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a92321fb3cf4b3f94e892795facd5ba'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:55.202406 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:55.213944 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 921/1822] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:55 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:55.214715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787321d4-2411-493d-8c73-96daf0a14a3b admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:55.215866 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-037dfbc9-38d1-45ee-aad1-c805c11c151e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a92321fb3cf4b3f94e892795facd5ba'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:55.217053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-787321d4-2411-493d-8c73-96daf0a14a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:55.226381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-037dfbc9-38d1-45ee-aad1-c805c11c151e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a92321fb3cf4b3f94e892795facd5ba'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:55.227273 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-037dfbc9-38d1-45ee-aad1-c805c11c151e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a92321fb3cf4b3f94e892795facd5ba'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:55.246244 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 902/1823] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:55 2022] DELETE /identity/v3/users/6a92321fb3cf4b3f94e892795facd5ba => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:55.292438 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787321d4-2411-493d-8c73-96daf0a14a3b admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:55.341741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037dfbc9-38d1-45ee-aad1-c805c11c151e admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:55.343691 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-037dfbc9-38d1-45ee-aad1-c805c11c151e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:55.347511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:55.367990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ea977-375f-4423-b0e6-6f865ad142b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:55.368651 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ea977-375f-4423-b0e6-6f865ad142b9 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:55.368927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ea977-375f-4423-b0e6-6f865ad142b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:55.369198 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ea977-375f-4423-b0e6-6f865ad142b9 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:55.369890 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ea977-375f-4423-b0e6-6f865ad142b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:55.371979 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ea977-375f-4423-b0e6-6f865ad142b9 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:55.401706 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 922/1824] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:55 2022] POST /identity/v3/users => generated 288 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:50:55.402727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ea977-375f-4423-b0e6-6f865ad142b9 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:55.414305 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ea977-375f-4423-b0e6-6f865ad142b9 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:55.419808 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c08c2-9087-4dd9-aaad-829ea79d1f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:55.420986 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c08c2-9087-4dd9-aaad-829ea79d1f7b admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:55.421389 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c08c2-9087-4dd9-aaad-829ea79d1f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:55.421666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c08c2-9087-4dd9-aaad-829ea79d1f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:55.422375 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c08c2-9087-4dd9-aaad-829ea79d1f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:55.423992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c08c2-9087-4dd9-aaad-829ea79d1f7b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:55.430938 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037dfbc9-38d1-45ee-aad1-c805c11c151e admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:55.436168 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 923/1825] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:55.436950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d9e79-7348-444f-95d6-0fa9edbe2174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:55.437502 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d9e79-7348-444f-95d6-0fa9edbe2174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:55.437610 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d9e79-7348-444f-95d6-0fa9edbe2174 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:55.437842 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d9e79-7348-444f-95d6-0fa9edbe2174 admin admin] PATH_INFO: `/v3/projects/e58fd3008c0c412495ac9ddf6decb5bf` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:55.439350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d9e79-7348-444f-95d6-0fa9edbe2174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e58fd3008c0c412495ac9ddf6decb5bf)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:55.442162 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d9e79-7348-444f-95d6-0fa9edbe2174 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:55.450537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c08c2-9087-4dd9-aaad-829ea79d1f7b admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:55.463881 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c08c2-9087-4dd9-aaad-829ea79d1f7b admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:55.471677 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17636bd-21d6-4d63-8bf6-5b1bb68a4a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:55.472262 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17636bd-21d6-4d63-8bf6-5b1bb68a4a25 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:55.472545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17636bd-21d6-4d63-8bf6-5b1bb68a4a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:55.472826 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17636bd-21d6-4d63-8bf6-5b1bb68a4a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:55.473467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17636bd-21d6-4d63-8bf6-5b1bb68a4a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:55.475500 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17636bd-21d6-4d63-8bf6-5b1bb68a4a25 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:55.493204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 924/1826] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:55.493984 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17636bd-21d6-4d63-8bf6-5b1bb68a4a25 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:55.504170 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-800d9e79-7348-444f-95d6-0fa9edbe2174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e58fd3008c0c412495ac9ddf6decb5bf', 'request_id': 'req-800d9e79-7348-444f-95d6-0fa9edbe2174'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:55.505056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17636bd-21d6-4d63-8bf6-5b1bb68a4a25 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:55.513548 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-800d9e79-7348-444f-95d6-0fa9edbe2174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e58fd3008c0c412495ac9ddf6decb5bf', 'request_id': 'req-800d9e79-7348-444f-95d6-0fa9edbe2174'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:55.521716 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-800d9e79-7348-444f-95d6-0fa9edbe2174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e58fd3008c0c412495ac9ddf6decb5bf', 'request_id': 'req-800d9e79-7348-444f-95d6-0fa9edbe2174'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:55.534038 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-800d9e79-7348-444f-95d6-0fa9edbe2174 admin admin] The token cache is being invalidate because project e58fd3008c0c412495ac9ddf6decb5bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:55.534328 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-800d9e79-7348-444f-95d6-0fa9edbe2174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:55.534994 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-800d9e79-7348-444f-95d6-0fa9edbe2174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:55.536895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 903/1827] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:55 2022] DELETE /identity/v3/projects/e58fd3008c0c412495ac9ddf6decb5bf => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:55.542754 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc08588-d0ca-4927-a43c-587991dabf32 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:55.543049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc08588-d0ca-4927-a43c-587991dabf32 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:55.543348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc08588-d0ca-4927-a43c-587991dabf32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:55.571233 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-4fc08588-d0ca-4927-a43c-587991dabf32 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:55.572499 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4fc08588-d0ca-4927-a43c-587991dabf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:55.573745 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e24c7a-1a45-4306-b365-853e9864a2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:55.574756 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e24c7a-1a45-4306-b365-853e9864a2d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:55.575407 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e24c7a-1a45-4306-b365-853e9864a2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:55.575821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e24c7a-1a45-4306-b365-853e9864a2d2 admin admin] PATH_INFO: `/v3/projects/660efc84ff32479fa3532a22325befe8/users/e176f65d65424a739fd2e79e231c6eb6/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:55.583841 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 904/1828] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:50:55.593805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309df706-bb70-4ec9-be12-d588a1151a5f None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:55.594060 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309df706-bb70-4ec9-be12-d588a1151a5f None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:55.594153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309df706-bb70-4ec9-be12-d588a1151a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:55.594540 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e24c7a-1a45-4306-b365-853e9864a2d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=660efc84ff32479fa3532a22325befe8, user_id=e176f65d65424a739fd2e79e231c6eb6, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:55.597832 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e24c7a-1a45-4306-b365-853e9864a2d2 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:55.619048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 925/1829] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:55 2022] PUT /identity/v3/projects/660efc84ff32479fa3532a22325befe8/users/e176f65d65424a739fd2e79e231c6eb6/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:55.622077 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-309df706-bb70-4ec9-be12-d588a1151a5f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:55.623272 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-309df706-bb70-4ec9-be12-d588a1151a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:55.625244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e24c7a-1a45-4306-b365-853e9864a2d2 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:55.627376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-23e24c7a-1a45-4306-b365-853e9864a2d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:55.680701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:55.701766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 905/1830] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:50:55.704320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc9df0-61f7-424f-a103-5520a6cea8ac None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:55.705108 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc9df0-61f7-424f-a103-5520a6cea8ac None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:55.705387 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc9df0-61f7-424f-a103-5520a6cea8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:55.713563 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e24c7a-1a45-4306-b365-853e9864a2d2 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:55.724055 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d84b32-c739-4a4f-a380-2905646b9dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:55.725030 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d84b32-c739-4a4f-a380-2905646b9dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:55.725324 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d84b32-c739-4a4f-a380-2905646b9dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:55.725770 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d84b32-c739-4a4f-a380-2905646b9dcc admin admin] PATH_INFO: `/v3/projects/69e10bd25c1e480d9524b8633d6dcdb3` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:55.727231 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d84b32-c739-4a4f-a380-2905646b9dcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=69e10bd25c1e480d9524b8633d6dcdb3)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:55.729176 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d84b32-c739-4a4f-a380-2905646b9dcc admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:55.741065 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-13bc9df0-61f7-424f-a103-5520a6cea8ac None None] MFA Rules not processed for user `e176f65d65424a739fd2e79e231c6eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:55.742317 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-13bc9df0-61f7-424f-a103-5520a6cea8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:55.784788 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:55.799452 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f4d84b32-c739-4a4f-a380-2905646b9dcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69e10bd25c1e480d9524b8633d6dcdb3', 'request_id': 'req-f4d84b32-c739-4a4f-a380-2905646b9dcc'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:55.812649 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f4d84b32-c739-4a4f-a380-2905646b9dcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69e10bd25c1e480d9524b8633d6dcdb3', 'request_id': 'req-f4d84b32-c739-4a4f-a380-2905646b9dcc'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:55.825953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 906/1831] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:55.828185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f4d84b32-c739-4a4f-a380-2905646b9dcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69e10bd25c1e480d9524b8633d6dcdb3', 'request_id': 'req-f4d84b32-c739-4a4f-a380-2905646b9dcc'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:55.830361 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc9df0-61f7-424f-a103-5520a6cea8ac None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:55.841833 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc9df0-61f7-424f-a103-5520a6cea8ac None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:55.843764 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f4d84b32-c739-4a4f-a380-2905646b9dcc admin admin] The token cache is being invalidate because project 69e10bd25c1e480d9524b8633d6dcdb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:55.844074 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f4d84b32-c739-4a4f-a380-2905646b9dcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:55.844748 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f4d84b32-c739-4a4f-a380-2905646b9dcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:55.846365 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 926/1832] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:55 2022] DELETE /identity/v3/projects/69e10bd25c1e480d9524b8633d6dcdb3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:55.847078 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d84b32-c739-4a4f-a380-2905646b9dcc admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:55.848421 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f4d84b32-c739-4a4f-a380-2905646b9dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:55.913345 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14f36d-6cfb-45af-8ec7-de7d8f631dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:55.914088 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14f36d-6cfb-45af-8ec7-de7d8f631dde admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:55.914370 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14f36d-6cfb-45af-8ec7-de7d8f631dde admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:55.914646 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14f36d-6cfb-45af-8ec7-de7d8f631dde admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:55.915941 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14f36d-6cfb-45af-8ec7-de7d8f631dde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:55.916274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14f36d-6cfb-45af-8ec7-de7d8f631dde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:55.918729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c14f36d-6cfb-45af-8ec7-de7d8f631dde admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:55.919554 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d84b32-c739-4a4f-a380-2905646b9dcc admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:55.924150 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a8116-1e77-4af5-986c-b220f3f16718 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:55.924773 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a8116-1e77-4af5-986c-b220f3f16718 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:55.925053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a8116-1e77-4af5-986c-b220f3f16718 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:55.925324 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a8116-1e77-4af5-986c-b220f3f16718 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:55.927117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-241a8116-1e77-4af5-986c-b220f3f16718 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:55.936180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 907/1833] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:55.936180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d99299-1fa4-4f8e-9b7c-fb552e7e81bf None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:55.936180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d99299-1fa4-4f8e-9b7c-fb552e7e81bf None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:55.936180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d99299-1fa4-4f8e-9b7c-fb552e7e81bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:55.958022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-b6d99299-1fa4-4f8e-9b7c-fb552e7e81bf None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:55.959481 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b6d99299-1fa4-4f8e-9b7c-fb552e7e81bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:55.969318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 908/1834] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:55.969991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d99299-1fa4-4f8e-9b7c-fb552e7e81bf None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:55.971495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b6d99299-1fa4-4f8e-9b7c-fb552e7e81bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:55.985243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d99299-1fa4-4f8e-9b7c-fb552e7e81bf None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:55.991116 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d731254-d937-45d3-a8d0-4c556239ac2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:55.991873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d731254-d937-45d3-a8d0-4c556239ac2a admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:55.992307 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d731254-d937-45d3-a8d0-4c556239ac2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:55.992587 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d731254-d937-45d3-a8d0-4c556239ac2a admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:55.993319 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d731254-d937-45d3-a8d0-4c556239ac2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:55.995253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d731254-d937-45d3-a8d0-4c556239ac2a admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.001079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a8116-1e77-4af5-986c-b220f3f16718 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.004999 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a8116-1e77-4af5-986c-b220f3f16718 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.015416 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 927/1835] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:56.016920 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046cf12-a33a-43d0-81cd-8d1f3ea9b9a3 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.017293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046cf12-a33a-43d0-81cd-8d1f3ea9b9a3 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.017623 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046cf12-a33a-43d0-81cd-8d1f3ea9b9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.041064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 909/1836] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:55 2022] POST /identity/v3/projects => generated 381 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:56.043856 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-e046cf12-a33a-43d0-81cd-8d1f3ea9b9a3 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:56.045108 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e046cf12-a33a-43d0-81cd-8d1f3ea9b9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:56.046199 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d731254-d937-45d3-a8d0-4c556239ac2a admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:56.113934 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:56.125956 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 928/1837] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:56.130482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046cf12-a33a-43d0-81cd-8d1f3ea9b9a3 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:56.131167 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d731254-d937-45d3-a8d0-4c556239ac2a admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:56.138736 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f95855-f24a-463e-8e4d-3b771da2ca25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:56.140380 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f95855-f24a-463e-8e4d-3b771da2ca25 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.140787 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046cf12-a33a-43d0-81cd-8d1f3ea9b9a3 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:56.141163 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f95855-f24a-463e-8e4d-3b771da2ca25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.141449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f95855-f24a-463e-8e4d-3b771da2ca25 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.143175 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f95855-f24a-463e-8e4d-3b771da2ca25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.145720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20135892-efda-46f2-9369-7c77402c1df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:56.146068 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f95855-f24a-463e-8e4d-3b771da2ca25 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.146454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20135892-efda-46f2-9369-7c77402c1df5 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.146741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20135892-efda-46f2-9369-7c77402c1df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.147006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20135892-efda-46f2-9369-7c77402c1df5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.147713 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20135892-efda-46f2-9369-7c77402c1df5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:56.148013 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20135892-efda-46f2-9369-7c77402c1df5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.149642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20135892-efda-46f2-9369-7c77402c1df5 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.162472 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 929/1838] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:56.166720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20135892-efda-46f2-9369-7c77402c1df5 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:56.178939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20135892-efda-46f2-9369-7c77402c1df5 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:56.187210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390b613-0662-4b30-b534-37a6b295b13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:56.187406 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 910/1839] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:56 2022] POST /identity/v3/users => generated 291 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:56.187526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390b613-0662-4b30-b534-37a6b295b13f admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.187526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390b613-0662-4b30-b534-37a6b295b13f admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.187526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390b613-0662-4b30-b534-37a6b295b13f admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.187526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390b613-0662-4b30-b534-37a6b295b13f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.189628 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f95855-f24a-463e-8e4d-3b771da2ca25 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:56.189789 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390b613-0662-4b30-b534-37a6b295b13f admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.195136 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:56.207000 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f95855-f24a-463e-8e4d-3b771da2ca25 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:56.215638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ad653-2823-4a18-adba-004e4ffa98b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:56.216325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ad653-2823-4a18-adba-004e4ffa98b8 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.216626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ad653-2823-4a18-adba-004e4ffa98b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.216898 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ad653-2823-4a18-adba-004e4ffa98b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.218283 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ad653-2823-4a18-adba-004e4ffa98b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.220136 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ad653-2823-4a18-adba-004e4ffa98b8 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.236501 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 930/1840] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:56 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:50:56.236706 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 911/1841] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:56.238851 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390b613-0662-4b30-b534-37a6b295b13f admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:56.240073 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ad653-2823-4a18-adba-004e4ffa98b8 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:56.252778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ad653-2823-4a18-adba-004e4ffa98b8 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:56.340000 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390b613-0662-4b30-b534-37a6b295b13f admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:56.343995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1909371-bc62-41b9-8d2a-cc73f4fc3554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:56.344870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1909371-bc62-41b9-8d2a-cc73f4fc3554 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.345154 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1909371-bc62-41b9-8d2a-cc73f4fc3554 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.345434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1909371-bc62-41b9-8d2a-cc73f4fc3554 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.346161 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63931f87-a370-4637-a7ee-9adeabefd45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:56.346285 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1909371-bc62-41b9-8d2a-cc73f4fc3554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.346888 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63931f87-a370-4637-a7ee-9adeabefd45b admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.347236 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63931f87-a370-4637-a7ee-9adeabefd45b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.347583 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63931f87-a370-4637-a7ee-9adeabefd45b admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.347927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1909371-bc62-41b9-8d2a-cc73f4fc3554 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.348536 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63931f87-a370-4637-a7ee-9adeabefd45b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.350600 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63931f87-a370-4637-a7ee-9adeabefd45b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.376147 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 912/1842] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:56.379250 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1909371-bc62-41b9-8d2a-cc73f4fc3554 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:56.391942 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1909371-bc62-41b9-8d2a-cc73f4fc3554 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:56.397450 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca6b2f-6e2c-4aa0-bef2-9ce209e17a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:56.398371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca6b2f-6e2c-4aa0-bef2-9ce209e17a7e admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.398727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca6b2f-6e2c-4aa0-bef2-9ce209e17a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.399074 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca6b2f-6e2c-4aa0-bef2-9ce209e17a7e admin admin] PATH_INFO: `/v3/projects/f67e0038551e4b4fb396eae8fd731543/users/c64d079aa3fd42c8b54e94a5c06682ac/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.404729 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 931/1843] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:56 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:56.407111 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63931f87-a370-4637-a7ee-9adeabefd45b admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:56.429709 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63931f87-a370-4637-a7ee-9adeabefd45b admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:56.431074 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca6b2f-6e2c-4aa0-bef2-9ce209e17a7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f67e0038551e4b4fb396eae8fd731543, user_id=c64d079aa3fd42c8b54e94a5c06682ac, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.435167 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22d62d-5d6b-4a88-93a3-a5e2a5f72bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:56.435679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca6b2f-6e2c-4aa0-bef2-9ce209e17a7e admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.435805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22d62d-5d6b-4a88-93a3-a5e2a5f72bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.436059 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22d62d-5d6b-4a88-93a3-a5e2a5f72bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.436604 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22d62d-5d6b-4a88-93a3-a5e2a5f72bb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.437234 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22d62d-5d6b-4a88-93a3-a5e2a5f72bb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.438839 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22d62d-5d6b-4a88-93a3-a5e2a5f72bb7 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.454524 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 932/1844] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:50:56.462248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22d62d-5d6b-4a88-93a3-a5e2a5f72bb7 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:56.473400 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 913/1845] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:56 2022] PUT /identity/v3/projects/f67e0038551e4b4fb396eae8fd731543/users/c64d079aa3fd42c8b54e94a5c06682ac/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:56.478988 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22d62d-5d6b-4a88-93a3-a5e2a5f72bb7 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:56.486120 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb6478-ac3f-4d8c-8870-4886055523a7 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.486411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb6478-ac3f-4d8c-8870-4886055523a7 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.486743 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb6478-ac3f-4d8c-8870-4886055523a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.519883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-29bb6478-ac3f-4d8c-8870-4886055523a7 None None] MFA Rules not processed for user `c64d079aa3fd42c8b54e94a5c06682ac`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:56.521158 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-29bb6478-ac3f-4d8c-8870-4886055523a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:56.553966 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debed723-915a-49c1-b802-5cf93a7247fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:56.555800 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debed723-915a-49c1-b802-5cf93a7247fb admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.556237 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debed723-915a-49c1-b802-5cf93a7247fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.556581 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debed723-915a-49c1-b802-5cf93a7247fb admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.557767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debed723-915a-49c1-b802-5cf93a7247fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.559697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debed723-915a-49c1-b802-5cf93a7247fb admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.575207 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 933/1846] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:50:56.577788 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debed723-915a-49c1-b802-5cf93a7247fb admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:56.588893 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debed723-915a-49c1-b802-5cf93a7247fb admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:56.595227 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a797c3-639f-440b-b311-7e23d96dd9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:56.595892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a797c3-639f-440b-b311-7e23d96dd9e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.596205 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a797c3-639f-440b-b311-7e23d96dd9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.596492 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a797c3-639f-440b-b311-7e23d96dd9e5 admin admin] PATH_INFO: `/v3/projects/0f4d1f88cc154bed92bf411b150f778b/users/794872f799184d9b84df099e17570109/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.607517 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 914/1847] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:56.612160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb6478-ac3f-4d8c-8870-4886055523a7 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:56.614515 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a797c3-639f-440b-b311-7e23d96dd9e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f4d1f88cc154bed92bf411b150f778b, user_id=794872f799184d9b84df099e17570109, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.617817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a797c3-639f-440b-b311-7e23d96dd9e5 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.622448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb6478-ac3f-4d8c-8870-4886055523a7 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:56.628774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5085e5b-70ff-40c6-801e-25468a3a2656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:56.629386 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5085e5b-70ff-40c6-801e-25468a3a2656 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.629668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5085e5b-70ff-40c6-801e-25468a3a2656 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.630042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5085e5b-70ff-40c6-801e-25468a3a2656 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.631001 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5085e5b-70ff-40c6-801e-25468a3a2656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.633021 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5085e5b-70ff-40c6-801e-25468a3a2656 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.635981 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 934/1848] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:56 2022] PUT /identity/v3/projects/0f4d1f88cc154bed92bf411b150f778b/users/794872f799184d9b84df099e17570109/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:56.647091 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d879c9a-2aa8-4e94-8c27-1865368d1ba0 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.647295 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d879c9a-2aa8-4e94-8c27-1865368d1ba0 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.647385 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d879c9a-2aa8-4e94-8c27-1865368d1ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.668744 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 915/1849] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:56 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:50:56.670185 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5085e5b-70ff-40c6-801e-25468a3a2656 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:56.676892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-6d879c9a-2aa8-4e94-8c27-1865368d1ba0 None None] MFA Rules not processed for user `794872f799184d9b84df099e17570109`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:56.678148 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-6d879c9a-2aa8-4e94-8c27-1865368d1ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:56.679432 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:56.706429 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:56.737438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5085e5b-70ff-40c6-801e-25468a3a2656 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:56.742062 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dab29-2af0-4895-a621-e0f43b9c0bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:56.742693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dab29-2af0-4895-a621-e0f43b9c0bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.742960 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dab29-2af0-4895-a621-e0f43b9c0bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.743232 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dab29-2af0-4895-a621-e0f43b9c0bf0 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.743861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761dab29-2af0-4895-a621-e0f43b9c0bf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.745816 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761dab29-2af0-4895-a621-e0f43b9c0bf0 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.749777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 935/1850] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:56.754188 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d879c9a-2aa8-4e94-8c27-1865368d1ba0 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:56.766410 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d879c9a-2aa8-4e94-8c27-1865368d1ba0 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:56.768710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 916/1851] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:56 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:50:56.772517 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dab29-2af0-4895-a621-e0f43b9c0bf0 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:56.774003 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864949ad-3c94-4dc5-8ee6-5cef69bb1eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:56.774643 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864949ad-3c94-4dc5-8ee6-5cef69bb1eac admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.774925 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864949ad-3c94-4dc5-8ee6-5cef69bb1eac admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.775201 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864949ad-3c94-4dc5-8ee6-5cef69bb1eac admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.775837 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864949ad-3c94-4dc5-8ee6-5cef69bb1eac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.778079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864949ad-3c94-4dc5-8ee6-5cef69bb1eac admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.782496 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dab29-2af0-4895-a621-e0f43b9c0bf0 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:56.788939 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1f822-95dc-45fc-89c4-7f66e4a059d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:56.789565 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1f822-95dc-45fc-89c4-7f66e4a059d0 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.790020 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1f822-95dc-45fc-89c4-7f66e4a059d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.790349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1f822-95dc-45fc-89c4-7f66e4a059d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.791429 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1f822-95dc-45fc-89c4-7f66e4a059d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.793080 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1f822-95dc-45fc-89c4-7f66e4a059d0 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.809086 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 917/1852] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:50:56.809289 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 936/1853] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:56 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:50:56.810633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1f822-95dc-45fc-89c4-7f66e4a059d0 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:56.813383 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864949ad-3c94-4dc5-8ee6-5cef69bb1eac admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:56.822247 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1f822-95dc-45fc-89c4-7f66e4a059d0 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:56.879119 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864949ad-3c94-4dc5-8ee6-5cef69bb1eac admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:56.879430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422300c-ded1-4aa7-a0be-c9e5b9f3f132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:56.880197 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422300c-ded1-4aa7-a0be-c9e5b9f3f132 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.880488 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422300c-ded1-4aa7-a0be-c9e5b9f3f132 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.880771 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422300c-ded1-4aa7-a0be-c9e5b9f3f132 admin admin] PATH_INFO: `/v3/projects/087bcb9cc16845738e6519e0bb2ce791/users/39e4353977e24bff8141d0f464dd8853/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.883794 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141861a-9f8a-4982-9856-8a7be8c611f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:56.884448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141861a-9f8a-4982-9856-8a7be8c611f4 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.884734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141861a-9f8a-4982-9856-8a7be8c611f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.885010 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141861a-9f8a-4982-9856-8a7be8c611f4 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.885645 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c141861a-9f8a-4982-9856-8a7be8c611f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.887576 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c141861a-9f8a-4982-9856-8a7be8c611f4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.897070 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422300c-ded1-4aa7-a0be-c9e5b9f3f132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=087bcb9cc16845738e6519e0bb2ce791, user_id=39e4353977e24bff8141d0f464dd8853, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.900888 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422300c-ded1-4aa7-a0be-c9e5b9f3f132 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.912061 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 937/1854] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:56 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:50:56.913150 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141861a-9f8a-4982-9856-8a7be8c611f4 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:56.915503 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 918/1855] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:56 2022] PUT /identity/v3/projects/087bcb9cc16845738e6519e0bb2ce791/users/39e4353977e24bff8141d0f464dd8853/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:56.919325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422300c-ded1-4aa7-a0be-c9e5b9f3f132 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:56.923309 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141861a-9f8a-4982-9856-8a7be8c611f4 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:56.983168 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422300c-ded1-4aa7-a0be-c9e5b9f3f132 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:56.984182 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba8c1b-9198-4709-935a-23c31a0b7091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:56.984808 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba8c1b-9198-4709-935a-23c31a0b7091 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.985093 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba8c1b-9198-4709-935a-23c31a0b7091 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.985372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba8c1b-9198-4709-935a-23c31a0b7091 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.986287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba8c1b-9198-4709-935a-23c31a0b7091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.988157 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba8c1b-9198-4709-935a-23c31a0b7091 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:56.988637 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618228c-c258-4e4d-99f6-1a230d4b4159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:56.989702 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618228c-c258-4e4d-99f6-1a230d4b4159 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:56.990205 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618228c-c258-4e4d-99f6-1a230d4b4159 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:56.990691 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618228c-c258-4e4d-99f6-1a230d4b4159 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:56.991739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618228c-c258-4e4d-99f6-1a230d4b4159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:56.993361 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618228c-c258-4e4d-99f6-1a230d4b4159 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:57.001263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 938/1856] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:57.005505 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba8c1b-9198-4709-935a-23c31a0b7091 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:57.010049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 919/1857] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:50:57.011260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618228c-c258-4e4d-99f6-1a230d4b4159 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:57.016409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba8c1b-9198-4709-935a-23c31a0b7091 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:57.019069 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:57.021614 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858b2d9-ee06-4fb9-b355-e42e708cd7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:57.022270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858b2d9-ee06-4fb9-b355-e42e708cd7e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:57.022552 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858b2d9-ee06-4fb9-b355-e42e708cd7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:57.022826 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858b2d9-ee06-4fb9-b355-e42e708cd7e6 admin admin] PATH_INFO: `/v3/projects/54499775af2b42368144250c5997496f/users/ded25791daef43c1bf1e0a7d7e9de83e/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:57.022957 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618228c-c258-4e4d-99f6-1a230d4b4159 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:57.028490 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8859b-1544-471f-9df8-2f7a0008f398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:57.029135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8859b-1544-471f-9df8-2f7a0008f398 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:57.029417 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8859b-1544-471f-9df8-2f7a0008f398 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:57.029698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8859b-1544-471f-9df8-2f7a0008f398 admin admin] PATH_INFO: `/v3/projects/087bcb9cc16845738e6519e0bb2ce791/users/39e4353977e24bff8141d0f464dd8853/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:57.032315 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8859b-1544-471f-9df8-2f7a0008f398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=087bcb9cc16845738e6519e0bb2ce791, user_id=39e4353977e24bff8141d0f464dd8853, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:57.034967 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8859b-1544-471f-9df8-2f7a0008f398 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:57.039120 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858b2d9-ee06-4fb9-b355-e42e708cd7e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54499775af2b42368144250c5997496f, user_id=ded25791daef43c1bf1e0a7d7e9de83e, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:57.041796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858b2d9-ee06-4fb9-b355-e42e708cd7e6 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:57.045238 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 920/1858] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:57 2022] PUT /identity/v3/projects/087bcb9cc16845738e6519e0bb2ce791/users/39e4353977e24bff8141d0f464dd8853/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:57.054822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 939/1859] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:57 2022] PUT /identity/v3/projects/54499775af2b42368144250c5997496f/users/ded25791daef43c1bf1e0a7d7e9de83e/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:57.056898 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dba2b4-f54d-4ca9-bff8-51bf969bb59a None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:57.057184 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dba2b4-f54d-4ca9-bff8-51bf969bb59a None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:57.057776 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dba2b4-f54d-4ca9-bff8-51bf969bb59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:57.058650 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858b2d9-ee06-4fb9-b355-e42e708cd7e6 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:57.065759 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:57.090904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-b8dba2b4-f54d-4ca9-bff8-51bf969bb59a None None] MFA Rules not processed for user `39e4353977e24bff8141d0f464dd8853`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:57.093213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b8dba2b4-f54d-4ca9-bff8-51bf969bb59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:57.129097 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858b2d9-ee06-4fb9-b355-e42e708cd7e6 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:57.133928 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551562af-5982-4a54-805c-f60ff2950104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:57.134545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551562af-5982-4a54-805c-f60ff2950104 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:57.134827 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551562af-5982-4a54-805c-f60ff2950104 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:57.135109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551562af-5982-4a54-805c-f60ff2950104 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:57.135992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551562af-5982-4a54-805c-f60ff2950104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:57.137526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551562af-5982-4a54-805c-f60ff2950104 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:57.152290 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 940/1860] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:50:57.154391 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551562af-5982-4a54-805c-f60ff2950104 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:57.167552 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551562af-5982-4a54-805c-f60ff2950104 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:57.173673 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a696943-db26-4346-866d-c29a3c027b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:57.174440 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a696943-db26-4346-866d-c29a3c027b5d admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:57.174812 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a696943-db26-4346-866d-c29a3c027b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:57.175174 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a696943-db26-4346-866d-c29a3c027b5d admin admin] PATH_INFO: `/v3/projects/54499775af2b42368144250c5997496f/users/ded25791daef43c1bf1e0a7d7e9de83e/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:57.175467 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 921/1861] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:57.177425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a696943-db26-4346-866d-c29a3c027b5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=54499775af2b42368144250c5997496f, user_id=ded25791daef43c1bf1e0a7d7e9de83e, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:57.182545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a696943-db26-4346-866d-c29a3c027b5d admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:57.187038 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dba2b4-f54d-4ca9-bff8-51bf969bb59a None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:57.195244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 941/1862] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:57 2022] PUT /identity/v3/projects/54499775af2b42368144250c5997496f/users/ded25791daef43c1bf1e0a7d7e9de83e/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:57.205864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff4bc0-53ab-48f6-809c-7b4784389308 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:57.206143 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff4bc0-53ab-48f6-809c-7b4784389308 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:57.206416 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff4bc0-53ab-48f6-809c-7b4784389308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:57.237640 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-4cff4bc0-53ab-48f6-809c-7b4784389308 None None] MFA Rules not processed for user `ded25791daef43c1bf1e0a7d7e9de83e`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:57.239030 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4cff4bc0-53ab-48f6-809c-7b4784389308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:57.259258 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dba2b4-f54d-4ca9-bff8-51bf969bb59a None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:57.318536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101075af-0848-4f0f-aab0-000170cc4a6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:57.319169 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101075af-0848-4f0f-aab0-000170cc4a6c service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:57.319492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101075af-0848-4f0f-aab0-000170cc4a6c service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:57.319784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101075af-0848-4f0f-aab0-000170cc4a6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:57.321346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 942/1863] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:57.332309 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff4bc0-53ab-48f6-809c-7b4784389308 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:57.333743 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4cff4bc0-53ab-48f6-809c-7b4784389308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:57.346543 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff4bc0-53ab-48f6-809c-7b4784389308 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:57.350845 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f07096-1657-4ffc-beb1-c89af9eefcc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:57.351544 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f07096-1657-4ffc-beb1-c89af9eefcc3 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:57.351984 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f07096-1657-4ffc-beb1-c89af9eefcc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:57.352319 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f07096-1657-4ffc-beb1-c89af9eefcc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:57.375548 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:57.393429 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101075af-0848-4f0f-aab0-000170cc4a6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:57.394908 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101075af-0848-4f0f-aab0-000170cc4a6c service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:57.407644 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 922/1864] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:50:57.426543 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f07096-1657-4ffc-beb1-c89af9eefcc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:57.427984 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f07096-1657-4ffc-beb1-c89af9eefcc3 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:57.440974 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 943/1865] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:50:57.992946 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101075af-0848-4f0f-aab0-000170cc4a6c service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:58.066743 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101075af-0848-4f0f-aab0-000170cc4a6c service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:58.071206 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7827cbe-881f-4ae5-8046-5e406c1e48dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:58.071827 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7827cbe-881f-4ae5-8046-5e406c1e48dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:58.072127 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7827cbe-881f-4ae5-8046-5e406c1e48dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:58.072402 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7827cbe-881f-4ae5-8046-5e406c1e48dd admin admin] PATH_INFO: `/v3/users/c64d079aa3fd42c8b54e94a5c06682ac` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:58.073503 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7827cbe-881f-4ae5-8046-5e406c1e48dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c64d079aa3fd42c8b54e94a5c06682ac)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:58.075413 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7827cbe-881f-4ae5-8046-5e406c1e48dd admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:58.121461 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a7827cbe-881f-4ae5-8046-5e406c1e48dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c64d079aa3fd42c8b54e94a5c06682ac'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:58.122120 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a7827cbe-881f-4ae5-8046-5e406c1e48dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c64d079aa3fd42c8b54e94a5c06682ac'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:58.134702 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a7827cbe-881f-4ae5-8046-5e406c1e48dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c64d079aa3fd42c8b54e94a5c06682ac'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:58.142261 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a7827cbe-881f-4ae5-8046-5e406c1e48dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c64d079aa3fd42c8b54e94a5c06682ac'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:58.142906 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a7827cbe-881f-4ae5-8046-5e406c1e48dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c64d079aa3fd42c8b54e94a5c06682ac'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:58.155502 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 923/1866] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:57 2022] DELETE /identity/v3/users/c64d079aa3fd42c8b54e94a5c06682ac => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:58.160345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96edff-b8cc-4874-9b4a-c869607c7549 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:58.160805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96edff-b8cc-4874-9b4a-c869607c7549 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:58.160895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96edff-b8cc-4874-9b4a-c869607c7549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:58.180870 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-ab96edff-b8cc-4874-9b4a-c869607c7549 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:58.182074 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ab96edff-b8cc-4874-9b4a-c869607c7549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:58.187653 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:58.242368 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 944/1867] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:58.253360 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7827cbe-881f-4ae5-8046-5e406c1e48dd admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:58.254822 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a7827cbe-881f-4ae5-8046-5e406c1e48dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:58.313763 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7827cbe-881f-4ae5-8046-5e406c1e48dd admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:58.317747 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c21b81-7d64-4c13-88e9-d3a028ecd7bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:58.318356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c21b81-7d64-4c13-88e9-d3a028ecd7bb service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:58.318635 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c21b81-7d64-4c13-88e9-d3a028ecd7bb service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:58.318907 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c21b81-7d64-4c13-88e9-d3a028ecd7bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:58.328134 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c21b81-7d64-4c13-88e9-d3a028ecd7bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:58.329571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c21b81-7d64-4c13-88e9-d3a028ecd7bb service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:58.339186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 924/1868] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:50:58.619642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96edff-b8cc-4874-9b4a-c869607c7549 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:58.621097 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ab96edff-b8cc-4874-9b4a-c869607c7549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:58.632843 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96edff-b8cc-4874-9b4a-c869607c7549 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:58.637572 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7e678-f45f-41ab-abf7-392508310a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:58.638209 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7e678-f45f-41ab-abf7-392508310a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:58.638487 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7e678-f45f-41ab-abf7-392508310a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:58.638757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7e678-f45f-41ab-abf7-392508310a34 admin admin] PATH_INFO: `/v3/projects/f67e0038551e4b4fb396eae8fd731543` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:58.639985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7e678-f45f-41ab-abf7-392508310a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f67e0038551e4b4fb396eae8fd731543)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:58.644298 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7e678-f45f-41ab-abf7-392508310a34 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:58.693312 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-67a7e678-f45f-41ab-abf7-392508310a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f67e0038551e4b4fb396eae8fd731543', 'request_id': 'req-67a7e678-f45f-41ab-abf7-392508310a34'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:58.700623 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-67a7e678-f45f-41ab-abf7-392508310a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f67e0038551e4b4fb396eae8fd731543', 'request_id': 'req-67a7e678-f45f-41ab-abf7-392508310a34'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:58.706937 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-67a7e678-f45f-41ab-abf7-392508310a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f67e0038551e4b4fb396eae8fd731543', 'request_id': 'req-67a7e678-f45f-41ab-abf7-392508310a34'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:58.719328 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-67a7e678-f45f-41ab-abf7-392508310a34 admin admin] The token cache is being invalidate because project f67e0038551e4b4fb396eae8fd731543 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:58.719830 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-67a7e678-f45f-41ab-abf7-392508310a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:58.720509 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-67a7e678-f45f-41ab-abf7-392508310a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:58.722079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 945/1869] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:58 2022] DELETE /identity/v3/projects/f67e0038551e4b4fb396eae8fd731543 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:58.726111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c21b81-7d64-4c13-88e9-d3a028ecd7bb service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:58.732069 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:58.736953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c21b81-7d64-4c13-88e9-d3a028ecd7bb service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:58.798550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036037ea-b075-4862-8065-ad6ff85d2046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:58.799195 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036037ea-b075-4862-8065-ad6ff85d2046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:58.799478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036037ea-b075-4862-8065-ad6ff85d2046 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:58.799755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036037ea-b075-4862-8065-ad6ff85d2046 admin admin] PATH_INFO: `/v3/users/39e4353977e24bff8141d0f464dd8853` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:58.800931 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036037ea-b075-4862-8065-ad6ff85d2046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39e4353977e24bff8141d0f464dd8853)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:58.802862 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036037ea-b075-4862-8065-ad6ff85d2046 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:58.852852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-036037ea-b075-4862-8065-ad6ff85d2046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39e4353977e24bff8141d0f464dd8853'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:58.853513 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-036037ea-b075-4862-8065-ad6ff85d2046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39e4353977e24bff8141d0f464dd8853'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:58.868998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-036037ea-b075-4862-8065-ad6ff85d2046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39e4353977e24bff8141d0f464dd8853'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:58.877361 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-036037ea-b075-4862-8065-ad6ff85d2046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39e4353977e24bff8141d0f464dd8853'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:58.878095 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-036037ea-b075-4862-8065-ad6ff85d2046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39e4353977e24bff8141d0f464dd8853'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:58.891515 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 925/1870] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:50:58 2022] DELETE /identity/v3/users/39e4353977e24bff8141d0f464dd8853 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:59.190645 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7e678-f45f-41ab-abf7-392508310a34 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:59.192769 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-67a7e678-f45f-41ab-abf7-392508310a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:59.206879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7e678-f45f-41ab-abf7-392508310a34 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:59.275823 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0cb58-0ffa-4fb6-82e7-2f685ff3135f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:59.276508 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0cb58-0ffa-4fb6-82e7-2f685ff3135f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:59.276877 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0cb58-0ffa-4fb6-82e7-2f685ff3135f admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:59.277164 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0cb58-0ffa-4fb6-82e7-2f685ff3135f admin admin] PATH_INFO: `/v3/projects/087bcb9cc16845738e6519e0bb2ce791` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:59.278480 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0cb58-0ffa-4fb6-82e7-2f685ff3135f admin admin] RBAC: Authorizing `identity:delete_project(project_id=087bcb9cc16845738e6519e0bb2ce791)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:59.281848 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0cb58-0ffa-4fb6-82e7-2f685ff3135f admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:59.324218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:59.337368 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f9c0cb58-0ffa-4fb6-82e7-2f685ff3135f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '087bcb9cc16845738e6519e0bb2ce791', 'request_id': 'req-f9c0cb58-0ffa-4fb6-82e7-2f685ff3135f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:59.343713 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f9c0cb58-0ffa-4fb6-82e7-2f685ff3135f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '087bcb9cc16845738e6519e0bb2ce791', 'request_id': 'req-f9c0cb58-0ffa-4fb6-82e7-2f685ff3135f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:59.349484 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f9c0cb58-0ffa-4fb6-82e7-2f685ff3135f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '087bcb9cc16845738e6519e0bb2ce791', 'request_id': 'req-f9c0cb58-0ffa-4fb6-82e7-2f685ff3135f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:59.361353 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f9c0cb58-0ffa-4fb6-82e7-2f685ff3135f admin admin] The token cache is being invalidate because project 087bcb9cc16845738e6519e0bb2ce791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:50:59.361641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f9c0cb58-0ffa-4fb6-82e7-2f685ff3135f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:59.362269 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f9c0cb58-0ffa-4fb6-82e7-2f685ff3135f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:50:59.363745 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 946/1871] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:50:59 2022] DELETE /identity/v3/projects/087bcb9cc16845738e6519e0bb2ce791 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:50:59.368518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb859f-03a6-40c7-8aa4-e2c60a195225 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:59.368967 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb859f-03a6-40c7-8aa4-e2c60a195225 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:59.369080 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb859f-03a6-40c7-8aa4-e2c60a195225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:59.388482 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-64fb859f-03a6-40c7-8aa4-e2c60a195225 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:59.389937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-64fb859f-03a6-40c7-8aa4-e2c60a195225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:59.450692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 926/1872] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:59.458599 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb368ce0-8145-4359-9cec-8aa07b8a527a None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:59.458896 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb368ce0-8145-4359-9cec-8aa07b8a527a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:59.459164 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb368ce0-8145-4359-9cec-8aa07b8a527a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:59.482582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-fb368ce0-8145-4359-9cec-8aa07b8a527a None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:59.484569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-fb368ce0-8145-4359-9cec-8aa07b8a527a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:59.492509 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 947/1873] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:59.496737 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb859f-03a6-40c7-8aa4-e2c60a195225 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:59.507141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb859f-03a6-40c7-8aa4-e2c60a195225 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:59.511633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67483f48-e4df-4741-9032-87cc0ccf8af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:59.512258 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67483f48-e4df-4741-9032-87cc0ccf8af2 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:59.512537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67483f48-e4df-4741-9032-87cc0ccf8af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:59.512806 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67483f48-e4df-4741-9032-87cc0ccf8af2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:59.513437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67483f48-e4df-4741-9032-87cc0ccf8af2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:50:59.513734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67483f48-e4df-4741-9032-87cc0ccf8af2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:59.515634 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67483f48-e4df-4741-9032-87cc0ccf8af2 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:59.527199 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 927/1874] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:50:59.532165 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb368ce0-8145-4359-9cec-8aa07b8a527a None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:59.541271 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb368ce0-8145-4359-9cec-8aa07b8a527a None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:59.546615 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ce21c-e77c-4be7-a67f-0a57ab8dc8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:59.547218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ce21c-e77c-4be7-a67f-0a57ab8dc8b9 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:59.547500 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ce21c-e77c-4be7-a67f-0a57ab8dc8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:59.547776 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ce21c-e77c-4be7-a67f-0a57ab8dc8b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:59.548472 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ce21c-e77c-4be7-a67f-0a57ab8dc8b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:59.550380 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ce21c-e77c-4be7-a67f-0a57ab8dc8b9 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:59.581047 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 948/1875] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:50:59 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:50:59.585148 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67483f48-e4df-4741-9032-87cc0ccf8af2 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:59.597915 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67483f48-e4df-4741-9032-87cc0ccf8af2 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:59.606599 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:50:59.674667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b5e05-575b-4414-a585-da6e132bdf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:59.675436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b5e05-575b-4414-a585-da6e132bdf24 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:59.675793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b5e05-575b-4414-a585-da6e132bdf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:59.676070 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b5e05-575b-4414-a585-da6e132bdf24 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:59.676772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b5e05-575b-4414-a585-da6e132bdf24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:59.678703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b5e05-575b-4414-a585-da6e132bdf24 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:59.706749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 928/1876] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:50:59 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:50:59.707977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ce21c-e77c-4be7-a67f-0a57ab8dc8b9 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:59.719370 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ce21c-e77c-4be7-a67f-0a57ab8dc8b9 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:59.724311 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec6aab-2f7e-45ac-89f8-e2e3c3cee410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:59.724920 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec6aab-2f7e-45ac-89f8-e2e3c3cee410 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:59.725200 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec6aab-2f7e-45ac-89f8-e2e3c3cee410 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:59.725472 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec6aab-2f7e-45ac-89f8-e2e3c3cee410 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:59.726114 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec6aab-2f7e-45ac-89f8-e2e3c3cee410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:59.727735 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec6aab-2f7e-45ac-89f8-e2e3c3cee410 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:59.741628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 949/1877] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:50:59.742931 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b5e05-575b-4414-a585-da6e132bdf24 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:59.754781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b5e05-575b-4414-a585-da6e132bdf24 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:59.759527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60025fd9-8a94-45fb-9237-93192ecc4175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:59.760043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60025fd9-8a94-45fb-9237-93192ecc4175 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:59.760442 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60025fd9-8a94-45fb-9237-93192ecc4175 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:59.760687 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60025fd9-8a94-45fb-9237-93192ecc4175 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:59.761332 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60025fd9-8a94-45fb-9237-93192ecc4175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:59.762952 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60025fd9-8a94-45fb-9237-93192ecc4175 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:59.776573 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 929/1878] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:50:59.781777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec6aab-2f7e-45ac-89f8-e2e3c3cee410 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:59.794858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec6aab-2f7e-45ac-89f8-e2e3c3cee410 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:50:59.804857 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740085fe-76af-4c52-9dfd-af27a3a092db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:50:59.805534 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740085fe-76af-4c52-9dfd-af27a3a092db admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:59.806110 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740085fe-76af-4c52-9dfd-af27a3a092db admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:59.806216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740085fe-76af-4c52-9dfd-af27a3a092db admin admin] PATH_INFO: `/v3/projects/2d6a4e760923417b958e27365564dc17/users/8a99aa8eafb4487ab539a3fa8199f626/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:59.825497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740085fe-76af-4c52-9dfd-af27a3a092db admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d6a4e760923417b958e27365564dc17, user_id=8a99aa8eafb4487ab539a3fa8199f626, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:50:59.828872 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740085fe-76af-4c52-9dfd-af27a3a092db admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:50:59.846680 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 950/1879] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:50:59 2022] PUT /identity/v3/projects/2d6a4e760923417b958e27365564dc17/users/8a99aa8eafb4487ab539a3fa8199f626/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:50:59.851822 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb5129-1522-47b5-a191-bf1931d09865 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:50:59.852151 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb5129-1522-47b5-a191-bf1931d09865 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:50:59.852428 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb5129-1522-47b5-a191-bf1931d09865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:50:59.880517 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-d4fb5129-1522-47b5-a191-bf1931d09865 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:50:59.883259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d4fb5129-1522-47b5-a191-bf1931d09865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:50:59.959785 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 930/1880] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:50:59.970532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740085fe-76af-4c52-9dfd-af27a3a092db admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:50:59.972437 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-740085fe-76af-4c52-9dfd-af27a3a092db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:00.065232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740085fe-76af-4c52-9dfd-af27a3a092db admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:00.069445 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937538dc-4838-4f53-8ebd-c41d10ffff54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:00.069975 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937538dc-4838-4f53-8ebd-c41d10ffff54 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:00.070386 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937538dc-4838-4f53-8ebd-c41d10ffff54 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:00.070551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937538dc-4838-4f53-8ebd-c41d10ffff54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:00.082185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937538dc-4838-4f53-8ebd-c41d10ffff54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:00.083789 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937538dc-4838-4f53-8ebd-c41d10ffff54 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:00.094736 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 951/1881] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:50:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:51:00.384177 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb5129-1522-47b5-a191-bf1931d09865 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:00.385719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d4fb5129-1522-47b5-a191-bf1931d09865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:00.401460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb5129-1522-47b5-a191-bf1931d09865 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:00.409023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294c3db-b9be-4353-a601-ceca96938812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:00.410738 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294c3db-b9be-4353-a601-ceca96938812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:00.411184 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294c3db-b9be-4353-a601-ceca96938812 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:00.411860 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294c3db-b9be-4353-a601-ceca96938812 admin admin] PATH_INFO: `/v3/users/e176f65d65424a739fd2e79e231c6eb6` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:00.413794 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294c3db-b9be-4353-a601-ceca96938812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e176f65d65424a739fd2e79e231c6eb6)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:00.418426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294c3db-b9be-4353-a601-ceca96938812 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:00.515612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a294c3db-b9be-4353-a601-ceca96938812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e176f65d65424a739fd2e79e231c6eb6'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:00.516411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a294c3db-b9be-4353-a601-ceca96938812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e176f65d65424a739fd2e79e231c6eb6'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:00.532730 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a294c3db-b9be-4353-a601-ceca96938812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e176f65d65424a739fd2e79e231c6eb6'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:00.545058 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a294c3db-b9be-4353-a601-ceca96938812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e176f65d65424a739fd2e79e231c6eb6'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:00.546564 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a294c3db-b9be-4353-a601-ceca96938812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e176f65d65424a739fd2e79e231c6eb6'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:00.566916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 931/1882] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:51:00 2022] DELETE /identity/v3/users/e176f65d65424a739fd2e79e231c6eb6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:00.572879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59c30d-d2b7-465f-84e4-07503ed4b867 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:00.573372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59c30d-d2b7-465f-84e4-07503ed4b867 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:00.573465 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59c30d-d2b7-465f-84e4-07503ed4b867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:00.596007 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-5f59c30d-d2b7-465f-84e4-07503ed4b867 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:00.597295 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-5f59c30d-d2b7-465f-84e4-07503ed4b867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:00.666759 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 952/1883] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:00.678122 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294c3db-b9be-4353-a601-ceca96938812 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:00.679942 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a294c3db-b9be-4353-a601-ceca96938812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:00.686311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:00.753903 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294c3db-b9be-4353-a601-ceca96938812 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:00.758001 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadcd9f0-03d6-4a8d-8f3f-697a423e7992 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:00.758716 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcd9f0-03d6-4a8d-8f3f-697a423e7992 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:00.758974 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcd9f0-03d6-4a8d-8f3f-697a423e7992 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:00.759249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcd9f0-03d6-4a8d-8f3f-697a423e7992 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:00.770368 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadcd9f0-03d6-4a8d-8f3f-697a423e7992 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:00.771947 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadcd9f0-03d6-4a8d-8f3f-697a423e7992 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:00.783774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 932/1884] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:51:00.933125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ff3b1-5ce9-45bf-92a3-4bb6d2e67fde None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:00.933410 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ff3b1-5ce9-45bf-92a3-4bb6d2e67fde None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:00.933689 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ff3b1-5ce9-45bf-92a3-4bb6d2e67fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:00.944207 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:00.988231 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-d93ff3b1-5ce9-45bf-92a3-4bb6d2e67fde None None] MFA Rules not processed for user `8a99aa8eafb4487ab539a3fa8199f626`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:00.989754 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d93ff3b1-5ce9-45bf-92a3-4bb6d2e67fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:01.119342 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 953/1885] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:01.121147 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadcd9f0-03d6-4a8d-8f3f-697a423e7992 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:01.123095 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-eadcd9f0-03d6-4a8d-8f3f-697a423e7992 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:01.143103 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadcd9f0-03d6-4a8d-8f3f-697a423e7992 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:01.149455 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaecd5a-d175-4a5d-937d-232b891c5416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:01.150548 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaecd5a-d175-4a5d-937d-232b891c5416 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:01.150910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaecd5a-d175-4a5d-937d-232b891c5416 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:01.151194 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaecd5a-d175-4a5d-937d-232b891c5416 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:01.151855 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaecd5a-d175-4a5d-937d-232b891c5416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:01.154529 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaecd5a-d175-4a5d-937d-232b891c5416 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:01.162656 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ff3b1-5ce9-45bf-92a3-4bb6d2e67fde None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:01.164136 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d93ff3b1-5ce9-45bf-92a3-4bb6d2e67fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:01.198642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ff3b1-5ce9-45bf-92a3-4bb6d2e67fde None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:01.208190 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a24a0d-8c6a-4859-8f70-7b08d7d7823c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:01.208771 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a24a0d-8c6a-4859-8f70-7b08d7d7823c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:01.209847 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a24a0d-8c6a-4859-8f70-7b08d7d7823c admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:01.210155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a24a0d-8c6a-4859-8f70-7b08d7d7823c admin admin] PATH_INFO: `/v3/projects/660efc84ff32479fa3532a22325befe8` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:01.211516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a24a0d-8c6a-4859-8f70-7b08d7d7823c admin admin] RBAC: Authorizing `identity:delete_project(project_id=660efc84ff32479fa3532a22325befe8)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:01.213977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a24a0d-8c6a-4859-8f70-7b08d7d7823c admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:01.220894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 933/1886] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:51:01 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:51:01.224862 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaecd5a-d175-4a5d-937d-232b891c5416 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:01.303035 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-b2a24a0d-8c6a-4859-8f70-7b08d7d7823c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '660efc84ff32479fa3532a22325befe8', 'request_id': 'req-b2a24a0d-8c6a-4859-8f70-7b08d7d7823c'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:01.345116 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-b2a24a0d-8c6a-4859-8f70-7b08d7d7823c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '660efc84ff32479fa3532a22325befe8', 'request_id': 'req-b2a24a0d-8c6a-4859-8f70-7b08d7d7823c'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:01.362182 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-b2a24a0d-8c6a-4859-8f70-7b08d7d7823c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '660efc84ff32479fa3532a22325befe8', 'request_id': 'req-b2a24a0d-8c6a-4859-8f70-7b08d7d7823c'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:01.376118 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-b2a24a0d-8c6a-4859-8f70-7b08d7d7823c admin admin] The token cache is being invalidate because project 660efc84ff32479fa3532a22325befe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:51:01.376478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-b2a24a0d-8c6a-4859-8f70-7b08d7d7823c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:01.377445 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-b2a24a0d-8c6a-4859-8f70-7b08d7d7823c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:01.380203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 954/1887] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:51:01 2022] DELETE /identity/v3/projects/660efc84ff32479fa3532a22325befe8 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:51:01.392334 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697d368-ae14-49a9-9178-5868c4246336 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:01.392334 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697d368-ae14-49a9-9178-5868c4246336 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:01.392334 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697d368-ae14-49a9-9178-5868c4246336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:01.400369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaecd5a-d175-4a5d-937d-232b891c5416 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:01.411381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-0697d368-ae14-49a9-9178-5868c4246336 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:01.412743 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-0697d368-ae14-49a9-9178-5868c4246336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:01.470740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5f87a-3924-407a-ae4e-caded950ef45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:01.471353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5f87a-3924-407a-ae4e-caded950ef45 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:01.471618 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5f87a-3924-407a-ae4e-caded950ef45 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:01.471962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5f87a-3924-407a-ae4e-caded950ef45 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:01.472846 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5f87a-3924-407a-ae4e-caded950ef45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:01.475164 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5f87a-3924-407a-ae4e-caded950ef45 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:01.487890 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 955/1888] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:01.499248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd14e2d-dee2-434e-9be3-3d17475de62f None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:01.499652 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd14e2d-dee2-434e-9be3-3d17475de62f None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:01.499953 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd14e2d-dee2-434e-9be3-3d17475de62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:01.510720 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 934/1889] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:51:01 2022] POST /identity/v3/users => generated 292 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:51:01.514429 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5f87a-3924-407a-ae4e-caded950ef45 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:01.516245 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-aaf5f87a-3924-407a-ae4e-caded950ef45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:01.522538 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-1dd14e2d-dee2-434e-9be3-3d17475de62f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:01.523744 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-1dd14e2d-dee2-434e-9be3-3d17475de62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:01.528705 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5f87a-3924-407a-ae4e-caded950ef45 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:01.533744 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 956/1890] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:01.534300 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6a360-c65c-4688-a849-1670f806d82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:01.534919 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6a360-c65c-4688-a849-1670f806d82f admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:01.535377 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6a360-c65c-4688-a849-1670f806d82f admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:01.535660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6a360-c65c-4688-a849-1670f806d82f admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:01.536339 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6a360-c65c-4688-a849-1670f806d82f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:01.537653 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd14e2d-dee2-434e-9be3-3d17475de62f None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:01.537973 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd6a360-c65c-4688-a849-1670f806d82f admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:01.548803 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd14e2d-dee2-434e-9be3-3d17475de62f None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:01.550387 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 935/1891] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:51:01.554213 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c8e49-be3a-4f1f-a912-35d2a6e824f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:01.554879 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c8e49-be3a-4f1f-a912-35d2a6e824f3 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:01.555001 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6a360-c65c-4688-a849-1670f806d82f admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:01.555163 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c8e49-be3a-4f1f-a912-35d2a6e824f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:01.555498 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c8e49-be3a-4f1f-a912-35d2a6e824f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:01.556206 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c8e49-be3a-4f1f-a912-35d2a6e824f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:51:01.556500 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c8e49-be3a-4f1f-a912-35d2a6e824f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:01.558094 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c8e49-be3a-4f1f-a912-35d2a6e824f3 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:01.566974 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:01.570996 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6a360-c65c-4688-a849-1670f806d82f admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:01.576920 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef578b-a80e-4e3c-9dc2-2a7e50e9c1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:01.577144 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 957/1892] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:51:01.578185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c8e49-be3a-4f1f-a912-35d2a6e824f3 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:01.578739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef578b-a80e-4e3c-9dc2-2a7e50e9c1b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:01.579121 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef578b-a80e-4e3c-9dc2-2a7e50e9c1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:01.579424 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef578b-a80e-4e3c-9dc2-2a7e50e9c1b3 admin admin] PATH_INFO: `/v3/projects/594b7a467b694173b87556ff59e3892a/users/aa9603c7063146c0bbcbfe7b9d233857/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:01.584470 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:01.589290 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c8e49-be3a-4f1f-a912-35d2a6e824f3 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:01.596833 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c935e6c-7c02-4c83-9aeb-c9a72082bdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:01.597526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c935e6c-7c02-4c83-9aeb-c9a72082bdc0 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:01.597900 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c935e6c-7c02-4c83-9aeb-c9a72082bdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:01.598169 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c935e6c-7c02-4c83-9aeb-c9a72082bdc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:01.598808 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c935e6c-7c02-4c83-9aeb-c9a72082bdc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:01.599686 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef578b-a80e-4e3c-9dc2-2a7e50e9c1b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=594b7a467b694173b87556ff59e3892a, user_id=aa9603c7063146c0bbcbfe7b9d233857, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:01.600796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c935e6c-7c02-4c83-9aeb-c9a72082bdc0 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:01.602718 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef578b-a80e-4e3c-9dc2-2a7e50e9c1b3 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:01.619154 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 936/1893] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:51:01 2022] PUT /identity/v3/projects/594b7a467b694173b87556ff59e3892a/users/aa9603c7063146c0bbcbfe7b9d233857/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:01.623287 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef578b-a80e-4e3c-9dc2-2a7e50e9c1b3 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:01.638290 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 958/1894] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:51:01 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:51:01.640380 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c935e6c-7c02-4c83-9aeb-c9a72082bdc0 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:01.700442 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef578b-a80e-4e3c-9dc2-2a7e50e9c1b3 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:01.707259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0cfe3d-4047-438c-9339-62bdf2a15167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:01.708304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cfe3d-4047-438c-9339-62bdf2a15167 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:01.708917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cfe3d-4047-438c-9339-62bdf2a15167 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:01.709459 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cfe3d-4047-438c-9339-62bdf2a15167 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:01.710399 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0cfe3d-4047-438c-9339-62bdf2a15167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:01.711110 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c935e6c-7c02-4c83-9aeb-c9a72082bdc0 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:01.712626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0cfe3d-4047-438c-9339-62bdf2a15167 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:01.716554 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc7bcc-158c-4661-99bb-8f237039f11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:01.717040 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc7bcc-158c-4661-99bb-8f237039f11f admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:01.717916 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc7bcc-158c-4661-99bb-8f237039f11f admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:01.718201 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc7bcc-158c-4661-99bb-8f237039f11f admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:01.719025 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc7bcc-158c-4661-99bb-8f237039f11f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:01.720968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc7bcc-158c-4661-99bb-8f237039f11f admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:01.725335 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 937/1895] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:51:01.729799 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0cfe3d-4047-438c-9339-62bdf2a15167 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:01.742471 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0cfe3d-4047-438c-9339-62bdf2a15167 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:01.747428 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d16cf0-9dd5-440c-a169-0f8dc3b097bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:01.747568 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 959/1896] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:51:01 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:51:01.748219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d16cf0-9dd5-440c-a169-0f8dc3b097bb admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:01.748337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc7bcc-158c-4661-99bb-8f237039f11f admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:01.748595 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d16cf0-9dd5-440c-a169-0f8dc3b097bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:01.749113 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d16cf0-9dd5-440c-a169-0f8dc3b097bb admin admin] PATH_INFO: `/v3/projects/594b7a467b694173b87556ff59e3892a/users/aa9603c7063146c0bbcbfe7b9d233857/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:01.751697 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d16cf0-9dd5-440c-a169-0f8dc3b097bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=594b7a467b694173b87556ff59e3892a, user_id=aa9603c7063146c0bbcbfe7b9d233857, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:01.754513 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d16cf0-9dd5-440c-a169-0f8dc3b097bb admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:01.759341 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc7bcc-158c-4661-99bb-8f237039f11f admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:01.766959 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 938/1897] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:51:01 2022] PUT /identity/v3/projects/594b7a467b694173b87556ff59e3892a/users/aa9603c7063146c0bbcbfe7b9d233857/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:01.840372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041e277-fca5-408a-9b93-69d415fe83ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:01.841203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041e277-fca5-408a-9b93-69d415fe83ce admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:01.841664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041e277-fca5-408a-9b93-69d415fe83ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:01.842007 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041e277-fca5-408a-9b93-69d415fe83ce admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:01.843337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041e277-fca5-408a-9b93-69d415fe83ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:01.845094 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041e277-fca5-408a-9b93-69d415fe83ce admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:01.859309 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 960/1898] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:51:01.863514 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d16cf0-9dd5-440c-a169-0f8dc3b097bb admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:01.891851 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d16cf0-9dd5-440c-a169-0f8dc3b097bb admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:01.898071 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0850b79-edad-468c-91d4-e3662c2732bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:01.898759 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0850b79-edad-468c-91d4-e3662c2732bf admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:01.899086 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0850b79-edad-468c-91d4-e3662c2732bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:01.899308 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0850b79-edad-468c-91d4-e3662c2732bf admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:01.900035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0850b79-edad-468c-91d4-e3662c2732bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:01.901928 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0850b79-edad-468c-91d4-e3662c2732bf admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:01.914931 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 939/1899] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:51:01.919967 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041e277-fca5-408a-9b93-69d415fe83ce admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:01.931100 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041e277-fca5-408a-9b93-69d415fe83ce admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:01.937512 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e27257-0e0b-4ead-8134-3bba3dd1bf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:01.938392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e27257-0e0b-4ead-8134-3bba3dd1bf05 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:01.938720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e27257-0e0b-4ead-8134-3bba3dd1bf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:01.939026 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e27257-0e0b-4ead-8134-3bba3dd1bf05 admin admin] PATH_INFO: `/v3/projects/cb6b323597d241daa50c1c053e4b389d/users/afd66831982e43208ac5231d9460f919/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:01.958612 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e27257-0e0b-4ead-8134-3bba3dd1bf05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6b323597d241daa50c1c053e4b389d, user_id=afd66831982e43208ac5231d9460f919, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:01.961345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e27257-0e0b-4ead-8134-3bba3dd1bf05 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:01.990495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 961/1900] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:51:01 2022] PUT /identity/v3/projects/cb6b323597d241daa50c1c053e4b389d/users/afd66831982e43208ac5231d9460f919/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:01.995296 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e25204-eab8-4365-b4ab-2e909899500f None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:01.995745 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e25204-eab8-4365-b4ab-2e909899500f None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:01.996160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e25204-eab8-4365-b4ab-2e909899500f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:02.026010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-02e25204-eab8-4365-b4ab-2e909899500f None None] MFA Rules not processed for user `afd66831982e43208ac5231d9460f919`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:02.027317 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-02e25204-eab8-4365-b4ab-2e909899500f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:02.096220 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 940/1901] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:51:02.108193 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e27257-0e0b-4ead-8134-3bba3dd1bf05 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:02.109920 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-38e27257-0e0b-4ead-8134-3bba3dd1bf05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:02.164753 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:02.183793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e27257-0e0b-4ead-8134-3bba3dd1bf05 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:02.187920 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0131e-1539-41a8-88ad-502f77d61ff3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:02.188682 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0131e-1539-41a8-88ad-502f77d61ff3 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:02.188964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0131e-1539-41a8-88ad-502f77d61ff3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:02.189237 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0131e-1539-41a8-88ad-502f77d61ff3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:02.199149 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0131e-1539-41a8-88ad-502f77d61ff3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:02.200467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0131e-1539-41a8-88ad-502f77d61ff3 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:02.211327 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 962/1902] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:51:02.665319 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4d46b-205d-48bf-ba83-fc0eca30f236 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:02.665319 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4d46b-205d-48bf-ba83-fc0eca30f236 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:02.665319 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4d46b-205d-48bf-ba83-fc0eca30f236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:02.688098 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-87f4d46b-205d-48bf-ba83-fc0eca30f236 None None] MFA Rules not processed for user `aa9603c7063146c0bbcbfe7b9d233857`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:02.689975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-87f4d46b-205d-48bf-ba83-fc0eca30f236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:02.735356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:02.777355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 941/1903] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:02.789561 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0131e-1539-41a8-88ad-502f77d61ff3 service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:02.798612 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0131e-1539-41a8-88ad-502f77d61ff3 service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:02.802333 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b41c2-34e0-4e52-b05e-36f96099fbdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:02.802933 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b41c2-34e0-4e52-b05e-36f96099fbdf service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:02.803211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b41c2-34e0-4e52-b05e-36f96099fbdf service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:02.803486 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b41c2-34e0-4e52-b05e-36f96099fbdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:02.805250 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-893b41c2-34e0-4e52-b05e-36f96099fbdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:02.868196 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b41c2-34e0-4e52-b05e-36f96099fbdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:02.869653 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b41c2-34e0-4e52-b05e-36f96099fbdf service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:02.879207 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 963/1904] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:51:03.006878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4d46b-205d-48bf-ba83-fc0eca30f236 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:03.008396 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-87f4d46b-205d-48bf-ba83-fc0eca30f236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:03.021510 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4d46b-205d-48bf-ba83-fc0eca30f236 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:03.025308 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c4583-881a-42a7-933a-2b703f748e6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:03.025878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c4583-881a-42a7-933a-2b703f748e6b service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:03.026153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c4583-881a-42a7-933a-2b703f748e6b service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:03.026426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c4583-881a-42a7-933a-2b703f748e6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:03.028287 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-1f3c4583-881a-42a7-933a-2b703f748e6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:03.098491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c4583-881a-42a7-933a-2b703f748e6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:03.100075 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c4583-881a-42a7-933a-2b703f748e6b service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:03.109717 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 942/1905] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:51:04.258082 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b41c2-34e0-4e52-b05e-36f96099fbdf service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:04.267751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b41c2-34e0-4e52-b05e-36f96099fbdf service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:04.271372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f389d-264d-4032-a933-faf9edb878f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:04.271965 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f389d-264d-4032-a933-faf9edb878f8 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:04.272274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f389d-264d-4032-a933-faf9edb878f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:04.272637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f389d-264d-4032-a933-faf9edb878f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:04.282504 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f389d-264d-4032-a933-faf9edb878f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:04.283927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f389d-264d-4032-a933-faf9edb878f8 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:04.293279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 964/1906] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:51:05.922258 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c4583-881a-42a7-933a-2b703f748e6b service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:05.985510 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c4583-881a-42a7-933a-2b703f748e6b service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:05.990090 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126bd7dd-d7a4-4d9c-a815-207acea09031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:05.990720 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bd7dd-d7a4-4d9c-a815-207acea09031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:05.991000 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bd7dd-d7a4-4d9c-a815-207acea09031 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:05.991305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bd7dd-d7a4-4d9c-a815-207acea09031 admin admin] PATH_INFO: `/v3/users/afd66831982e43208ac5231d9460f919` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:05.992461 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126bd7dd-d7a4-4d9c-a815-207acea09031 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afd66831982e43208ac5231d9460f919)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:05.994390 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126bd7dd-d7a4-4d9c-a815-207acea09031 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:06.058999 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-126bd7dd-d7a4-4d9c-a815-207acea09031 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afd66831982e43208ac5231d9460f919'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:06.060158 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-126bd7dd-d7a4-4d9c-a815-207acea09031 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afd66831982e43208ac5231d9460f919'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:06.080717 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-126bd7dd-d7a4-4d9c-a815-207acea09031 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afd66831982e43208ac5231d9460f919'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:06.083117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:06.092857 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-126bd7dd-d7a4-4d9c-a815-207acea09031 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afd66831982e43208ac5231d9460f919'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:06.094236 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-126bd7dd-d7a4-4d9c-a815-207acea09031 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afd66831982e43208ac5231d9460f919'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:06.109899 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 943/1907] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:51:05 2022] DELETE /identity/v3/users/afd66831982e43208ac5231d9460f919 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:06.114906 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eeb28c-5984-4388-b5d9-a42262f63095 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:06.115797 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eeb28c-5984-4388-b5d9-a42262f63095 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:06.116376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eeb28c-5984-4388-b5d9-a42262f63095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:06.139655 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-a1eeb28c-5984-4388-b5d9-a42262f63095 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:06.141059 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a1eeb28c-5984-4388-b5d9-a42262f63095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:06.206076 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 965/1908] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:51:06.219670 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126bd7dd-d7a4-4d9c-a815-207acea09031 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:06.221367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-126bd7dd-d7a4-4d9c-a815-207acea09031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:06.284961 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126bd7dd-d7a4-4d9c-a815-207acea09031 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:06.288729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd476906-f72c-4c14-b814-d484c86dc3f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:06.289347 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd476906-f72c-4c14-b814-d484c86dc3f4 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:06.289624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd476906-f72c-4c14-b814-d484c86dc3f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:06.289895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd476906-f72c-4c14-b814-d484c86dc3f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:06.301079 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd476906-f72c-4c14-b814-d484c86dc3f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:06.302794 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd476906-f72c-4c14-b814-d484c86dc3f4 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:06.314334 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 944/1909] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:51:06.586429 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eeb28c-5984-4388-b5d9-a42262f63095 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:06.588296 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a1eeb28c-5984-4388-b5d9-a42262f63095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:06.603790 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eeb28c-5984-4388-b5d9-a42262f63095 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:06.610609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d3051-051c-4906-8477-74959ab94a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:06.611255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d3051-051c-4906-8477-74959ab94a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:06.611532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d3051-051c-4906-8477-74959ab94a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:06.611804 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d3051-051c-4906-8477-74959ab94a56 admin admin] PATH_INFO: `/v3/projects/cb6b323597d241daa50c1c053e4b389d` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:06.612990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d3051-051c-4906-8477-74959ab94a56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb6b323597d241daa50c1c053e4b389d)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:06.614930 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d3051-051c-4906-8477-74959ab94a56 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:06.662621 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ee3d3051-051c-4906-8477-74959ab94a56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb6b323597d241daa50c1c053e4b389d', 'request_id': 'req-ee3d3051-051c-4906-8477-74959ab94a56'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:06.670331 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ee3d3051-051c-4906-8477-74959ab94a56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb6b323597d241daa50c1c053e4b389d', 'request_id': 'req-ee3d3051-051c-4906-8477-74959ab94a56'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:06.674843 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:06.678379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ee3d3051-051c-4906-8477-74959ab94a56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb6b323597d241daa50c1c053e4b389d', 'request_id': 'req-ee3d3051-051c-4906-8477-74959ab94a56'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:06.692375 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ee3d3051-051c-4906-8477-74959ab94a56 admin admin] The token cache is being invalidate because project cb6b323597d241daa50c1c053e4b389d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:51:06.692667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ee3d3051-051c-4906-8477-74959ab94a56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:06.693314 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ee3d3051-051c-4906-8477-74959ab94a56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:06.694806 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 966/1910] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:51:06 2022] DELETE /identity/v3/projects/cb6b323597d241daa50c1c053e4b389d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:08.226188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd476906-f72c-4c14-b814-d484c86dc3f4 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:08.227751 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-fd476906-f72c-4c14-b814-d484c86dc3f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:08.241838 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd476906-f72c-4c14-b814-d484c86dc3f4 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:08.303925 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15303cf3-8ade-4727-83eb-0c95d4d67f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:08.305007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15303cf3-8ade-4727-83eb-0c95d4d67f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:08.305007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15303cf3-8ade-4727-83eb-0c95d4d67f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:08.305322 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15303cf3-8ade-4727-83eb-0c95d4d67f23 admin admin] PATH_INFO: `/v3/users/0f1cded746ca4d4b93c57260b05eaa8b` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:08.306710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15303cf3-8ade-4727-83eb-0c95d4d67f23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f1cded746ca4d4b93c57260b05eaa8b)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:08.308753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15303cf3-8ade-4727-83eb-0c95d4d67f23 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:08.360693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-15303cf3-8ade-4727-83eb-0c95d4d67f23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f1cded746ca4d4b93c57260b05eaa8b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:08.361518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-15303cf3-8ade-4727-83eb-0c95d4d67f23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f1cded746ca4d4b93c57260b05eaa8b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:08.375144 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-15303cf3-8ade-4727-83eb-0c95d4d67f23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f1cded746ca4d4b93c57260b05eaa8b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:08.383042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-15303cf3-8ade-4727-83eb-0c95d4d67f23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f1cded746ca4d4b93c57260b05eaa8b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:08.383774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-15303cf3-8ade-4727-83eb-0c95d4d67f23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f1cded746ca4d4b93c57260b05eaa8b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:08.401892 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 945/1911] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:51:08 2022] DELETE /identity/v3/users/0f1cded746ca4d4b93c57260b05eaa8b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:08.406772 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6201ea-6305-40d4-a849-d8ba97ad69bb None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:08.407276 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6201ea-6305-40d4-a849-d8ba97ad69bb None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:08.407373 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6201ea-6305-40d4-a849-d8ba97ad69bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:08.428939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-4d6201ea-6305-40d4-a849-d8ba97ad69bb None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:08.430239 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4d6201ea-6305-40d4-a849-d8ba97ad69bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:08.514701 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 967/1912] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:08.524617 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15303cf3-8ade-4727-83eb-0c95d4d67f23 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:08.526142 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-15303cf3-8ade-4727-83eb-0c95d4d67f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:08.537449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:08.589531 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15303cf3-8ade-4727-83eb-0c95d4d67f23 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:08.593779 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0056f1-9016-4ce5-a782-819a1c570a93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:08.594384 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0056f1-9016-4ce5-a782-819a1c570a93 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:08.594667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0056f1-9016-4ce5-a782-819a1c570a93 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:08.594938 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0056f1-9016-4ce5-a782-819a1c570a93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:08.606601 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0056f1-9016-4ce5-a782-819a1c570a93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:08.608655 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0056f1-9016-4ce5-a782-819a1c570a93 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:08.618279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 946/1913] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:51:08.890290 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6201ea-6305-40d4-a849-d8ba97ad69bb None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:08.893167 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4d6201ea-6305-40d4-a849-d8ba97ad69bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:08.905855 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6201ea-6305-40d4-a849-d8ba97ad69bb None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:08.910455 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69c48a-7849-42bd-844d-09dba8f4af83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:08.911093 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69c48a-7849-42bd-844d-09dba8f4af83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:08.911372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69c48a-7849-42bd-844d-09dba8f4af83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:08.911764 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69c48a-7849-42bd-844d-09dba8f4af83 admin admin] PATH_INFO: `/v3/projects/a5f8ad5cd30d417bad9bebdf6fadbb97` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:08.913102 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69c48a-7849-42bd-844d-09dba8f4af83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5f8ad5cd30d417bad9bebdf6fadbb97)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:08.914929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69c48a-7849-42bd-844d-09dba8f4af83 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:08.962880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-bd69c48a-7849-42bd-844d-09dba8f4af83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5f8ad5cd30d417bad9bebdf6fadbb97', 'request_id': 'req-bd69c48a-7849-42bd-844d-09dba8f4af83'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:08.970663 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-bd69c48a-7849-42bd-844d-09dba8f4af83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5f8ad5cd30d417bad9bebdf6fadbb97', 'request_id': 'req-bd69c48a-7849-42bd-844d-09dba8f4af83'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:08.976484 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-bd69c48a-7849-42bd-844d-09dba8f4af83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5f8ad5cd30d417bad9bebdf6fadbb97', 'request_id': 'req-bd69c48a-7849-42bd-844d-09dba8f4af83'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:08.987975 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-bd69c48a-7849-42bd-844d-09dba8f4af83 admin admin] The token cache is being invalidate because project a5f8ad5cd30d417bad9bebdf6fadbb97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:51:08.988308 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-bd69c48a-7849-42bd-844d-09dba8f4af83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:08.988938 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-bd69c48a-7849-42bd-844d-09dba8f4af83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:08.990432 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 968/1914] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:51:08 2022] DELETE /identity/v3/projects/a5f8ad5cd30d417bad9bebdf6fadbb97 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:08.995280 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea828d-8a3c-431e-a5da-e1099cda8203 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:08.995561 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea828d-8a3c-431e-a5da-e1099cda8203 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:08.995833 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea828d-8a3c-431e-a5da-e1099cda8203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:09.016003 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-3fea828d-8a3c-431e-a5da-e1099cda8203 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:09.017240 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-3fea828d-8a3c-431e-a5da-e1099cda8203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:09.075974 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 947/1915] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:09.084224 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4252d-8d08-4218-89ec-b42bd6be372c None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:09.084515 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4252d-8d08-4218-89ec-b42bd6be372c None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:09.084793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4252d-8d08-4218-89ec-b42bd6be372c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:09.105147 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-8cf4252d-8d08-4218-89ec-b42bd6be372c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:09.106351 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-8cf4252d-8d08-4218-89ec-b42bd6be372c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:09.115079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 969/1916] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:09.119353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea828d-8a3c-431e-a5da-e1099cda8203 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:09.120173 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4252d-8d08-4218-89ec-b42bd6be372c None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:09.121489 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-8cf4252d-8d08-4218-89ec-b42bd6be372c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:09.129560 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea828d-8a3c-431e-a5da-e1099cda8203 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:09.133737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4252d-8d08-4218-89ec-b42bd6be372c None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:09.134250 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e82df8-e7d4-443a-8db2-bd1247680018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:09.134864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e82df8-e7d4-443a-8db2-bd1247680018 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:09.135166 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e82df8-e7d4-443a-8db2-bd1247680018 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:09.135425 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e82df8-e7d4-443a-8db2-bd1247680018 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:09.136073 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e82df8-e7d4-443a-8db2-bd1247680018 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:51:09.136410 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e82df8-e7d4-443a-8db2-bd1247680018 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:09.138050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e82df8-e7d4-443a-8db2-bd1247680018 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:09.138566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38612fdf-8212-4cec-89a7-48a482860e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:09.139216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38612fdf-8212-4cec-89a7-48a482860e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:09.139688 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38612fdf-8212-4cec-89a7-48a482860e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:09.140049 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38612fdf-8212-4cec-89a7-48a482860e4a admin admin] PATH_INFO: `/v3/users/794872f799184d9b84df099e17570109` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:09.141235 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38612fdf-8212-4cec-89a7-48a482860e4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=794872f799184d9b84df099e17570109)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:09.143130 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38612fdf-8212-4cec-89a7-48a482860e4a admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:09.149840 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 948/1917] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:51:09.154083 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e82df8-e7d4-443a-8db2-bd1247680018 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:09.164420 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e82df8-e7d4-443a-8db2-bd1247680018 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:09.171443 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b9f7c-4f10-4129-a98c-6489691553a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:09.172166 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b9f7c-4f10-4129-a98c-6489691553a0 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:09.172477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b9f7c-4f10-4129-a98c-6489691553a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:09.172744 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b9f7c-4f10-4129-a98c-6489691553a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:09.173367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b9f7c-4f10-4129-a98c-6489691553a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:09.175568 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b9f7c-4f10-4129-a98c-6489691553a0 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:09.194156 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-38612fdf-8212-4cec-89a7-48a482860e4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '794872f799184d9b84df099e17570109'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:09.194968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-38612fdf-8212-4cec-89a7-48a482860e4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '794872f799184d9b84df099e17570109'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:09.206332 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:DELETE=16,keystone:SELECT=83,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:09.206636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 949/1918] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:51:09 2022] POST /identity/v3/projects => generated 391 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:51:09.208908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-38612fdf-8212-4cec-89a7-48a482860e4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '794872f799184d9b84df099e17570109'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:09.212487 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b9f7c-4f10-4129-a98c-6489691553a0 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:09.214750 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-0a7b9f7c-4f10-4129-a98c-6489691553a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:09.218807 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-38612fdf-8212-4cec-89a7-48a482860e4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '794872f799184d9b84df099e17570109'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:09.227322 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-38612fdf-8212-4cec-89a7-48a482860e4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '794872f799184d9b84df099e17570109'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:09.234172 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 970/1919] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:51:09 2022] DELETE /identity/v3/users/794872f799184d9b84df099e17570109 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:51:09.238617 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada3777-764f-4bd8-a354-904959415279 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:09.238900 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada3777-764f-4bd8-a354-904959415279 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:09.239175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada3777-764f-4bd8-a354-904959415279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:09.258767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-9ada3777-764f-4bd8-a354-904959415279 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:09.259992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-9ada3777-764f-4bd8-a354-904959415279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:09.293773 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b9f7c-4f10-4129-a98c-6489691553a0 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:09.313655 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:09.323702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 971/1920] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:51:09.333755 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada3777-764f-4bd8-a354-904959415279 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:09.335177 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-9ada3777-764f-4bd8-a354-904959415279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:09.355699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152165af-7cc8-4f04-b966-5194c3def742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:09.356610 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152165af-7cc8-4f04-b966-5194c3def742 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:09.356920 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152165af-7cc8-4f04-b966-5194c3def742 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:09.357191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152165af-7cc8-4f04-b966-5194c3def742 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:09.357819 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152165af-7cc8-4f04-b966-5194c3def742 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:09.360316 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152165af-7cc8-4f04-b966-5194c3def742 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:09.384776 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 950/1921] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:51:09 2022] POST /identity/v3/users => generated 296 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:51:09.385818 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152165af-7cc8-4f04-b966-5194c3def742 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:09.395900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152165af-7cc8-4f04-b966-5194c3def742 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:09.398086 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada3777-764f-4bd8-a354-904959415279 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:09.400603 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724f119-9c89-430a-a46b-b6047106e310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:09.401264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724f119-9c89-430a-a46b-b6047106e310 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:09.401623 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724f119-9c89-430a-a46b-b6047106e310 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:09.401967 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724f119-9c89-430a-a46b-b6047106e310 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:09.402072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042637a0-fd26-45de-9c3f-4b63c6a02f8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:09.402627 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042637a0-fd26-45de-9c3f-4b63c6a02f8a service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:09.402800 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724f119-9c89-430a-a46b-b6047106e310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:09.402898 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042637a0-fd26-45de-9c3f-4b63c6a02f8a service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:09.403166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042637a0-fd26-45de-9c3f-4b63c6a02f8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:09.404623 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724f119-9c89-430a-a46b-b6047106e310 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:09.413388 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042637a0-fd26-45de-9c3f-4b63c6a02f8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:09.414795 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042637a0-fd26-45de-9c3f-4b63c6a02f8a service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:09.415574 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 951/1922] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:51:09.420041 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724f119-9c89-430a-a46b-b6047106e310 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:09.424835 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 972/1923] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:51:09.430169 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724f119-9c89-430a-a46b-b6047106e310 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:09.434898 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10331e-4b3c-47f8-b3fb-0a68e3505b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:09.435505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10331e-4b3c-47f8-b3fb-0a68e3505b02 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:09.435791 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10331e-4b3c-47f8-b3fb-0a68e3505b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:09.436135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10331e-4b3c-47f8-b3fb-0a68e3505b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:09.436955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10331e-4b3c-47f8-b3fb-0a68e3505b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:09.438577 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10331e-4b3c-47f8-b3fb-0a68e3505b02 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:09.450924 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 952/1924] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:51:09.455580 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042637a0-fd26-45de-9c3f-4b63c6a02f8a service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:09.470434 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042637a0-fd26-45de-9c3f-4b63c6a02f8a service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:09.477005 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98f70e-5e69-4c6f-9f4b-466883b0bf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:09.477731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98f70e-5e69-4c6f-9f4b-466883b0bf44 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:09.478111 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98f70e-5e69-4c6f-9f4b-466883b0bf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:09.478367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98f70e-5e69-4c6f-9f4b-466883b0bf44 admin admin] PATH_INFO: `/v3/projects/fffcab8b7320471b82c1d0f8d11153ed/users/abfafb58fd6f4b1eb3a904af79f39723/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:09.498262 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c98f70e-5e69-4c6f-9f4b-466883b0bf44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fffcab8b7320471b82c1d0f8d11153ed, user_id=abfafb58fd6f4b1eb3a904af79f39723, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:09.502732 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c98f70e-5e69-4c6f-9f4b-466883b0bf44 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:09.528737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 973/1925] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:51:09 2022] PUT /identity/v3/projects/fffcab8b7320471b82c1d0f8d11153ed/users/abfafb58fd6f4b1eb3a904af79f39723/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:51:09.532232 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe54c86-9e64-402b-aa4e-0d9b5d73857e None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:09.532523 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe54c86-9e64-402b-aa4e-0d9b5d73857e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:09.532800 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe54c86-9e64-402b-aa4e-0d9b5d73857e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:09.565950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-cfe54c86-9e64-402b-aa4e-0d9b5d73857e None None] MFA Rules not processed for user `abfafb58fd6f4b1eb3a904af79f39723`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:09.567772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-cfe54c86-9e64-402b-aa4e-0d9b5d73857e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:09.681058 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 953/1926] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:09.690400 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98f70e-5e69-4c6f-9f4b-466883b0bf44 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:09.693746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-3c98f70e-5e69-4c6f-9f4b-466883b0bf44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:09.771283 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe54c86-9e64-402b-aa4e-0d9b5d73857e None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:09.772957 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-cfe54c86-9e64-402b-aa4e-0d9b5d73857e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:09.792119 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98f70e-5e69-4c6f-9f4b-466883b0bf44 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:09.796525 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179f197-eb33-461e-bf29-5c8ea3785ab4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:09.799125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179f197-eb33-461e-bf29-5c8ea3785ab4 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:09.799691 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179f197-eb33-461e-bf29-5c8ea3785ab4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:09.800226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179f197-eb33-461e-bf29-5c8ea3785ab4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:09.813482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179f197-eb33-461e-bf29-5c8ea3785ab4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:09.815036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179f197-eb33-461e-bf29-5c8ea3785ab4 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:09.823216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:09.830071 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 974/1927] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:09 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:51:09.888213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe54c86-9e64-402b-aa4e-0d9b5d73857e None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:09.894297 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d5cb6-45fe-43bb-83fa-df63004225ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:09.895064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d5cb6-45fe-43bb-83fa-df63004225ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:09.895350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d5cb6-45fe-43bb-83fa-df63004225ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:09.895631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d5cb6-45fe-43bb-83fa-df63004225ad admin admin] PATH_INFO: `/v3/projects/0f4d1f88cc154bed92bf411b150f778b` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:09.896843 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d5cb6-45fe-43bb-83fa-df63004225ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f4d1f88cc154bed92bf411b150f778b)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:09.898760 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d5cb6-45fe-43bb-83fa-df63004225ad admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:09.924473 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:09.955264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-734d5cb6-45fe-43bb-83fa-df63004225ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f4d1f88cc154bed92bf411b150f778b', 'request_id': 'req-734d5cb6-45fe-43bb-83fa-df63004225ad'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:09.964801 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-734d5cb6-45fe-43bb-83fa-df63004225ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f4d1f88cc154bed92bf411b150f778b', 'request_id': 'req-734d5cb6-45fe-43bb-83fa-df63004225ad'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:09.972755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-734d5cb6-45fe-43bb-83fa-df63004225ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f4d1f88cc154bed92bf411b150f778b', 'request_id': 'req-734d5cb6-45fe-43bb-83fa-df63004225ad'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:09.984569 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-734d5cb6-45fe-43bb-83fa-df63004225ad admin admin] The token cache is being invalidate because project 0f4d1f88cc154bed92bf411b150f778b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:51:09.984883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-734d5cb6-45fe-43bb-83fa-df63004225ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:09.985608 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-734d5cb6-45fe-43bb-83fa-df63004225ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:09.987182 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 954/1928] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:51:09 2022] DELETE /identity/v3/projects/0f4d1f88cc154bed92bf411b150f778b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:51:09.992099 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179f197-eb33-461e-bf29-5c8ea3785ab4 service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:10.008858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179f197-eb33-461e-bf29-5c8ea3785ab4 service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:10.077646 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082aac85-7594-40e1-b060-bdcf0e7d5ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:10.078297 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082aac85-7594-40e1-b060-bdcf0e7d5ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:10.078627 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082aac85-7594-40e1-b060-bdcf0e7d5ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:10.079420 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082aac85-7594-40e1-b060-bdcf0e7d5ea5 admin admin] PATH_INFO: `/v3/users/ded25791daef43c1bf1e0a7d7e9de83e` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:10.080822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082aac85-7594-40e1-b060-bdcf0e7d5ea5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ded25791daef43c1bf1e0a7d7e9de83e)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:10.083609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082aac85-7594-40e1-b060-bdcf0e7d5ea5 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:10.168726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-082aac85-7594-40e1-b060-bdcf0e7d5ea5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ded25791daef43c1bf1e0a7d7e9de83e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:10.169448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-082aac85-7594-40e1-b060-bdcf0e7d5ea5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ded25791daef43c1bf1e0a7d7e9de83e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:10.190778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-082aac85-7594-40e1-b060-bdcf0e7d5ea5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ded25791daef43c1bf1e0a7d7e9de83e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:10.198077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-082aac85-7594-40e1-b060-bdcf0e7d5ea5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ded25791daef43c1bf1e0a7d7e9de83e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:10.207828 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-082aac85-7594-40e1-b060-bdcf0e7d5ea5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ded25791daef43c1bf1e0a7d7e9de83e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:10.214248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 975/1929] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:51:09 2022] DELETE /identity/v3/users/ded25791daef43c1bf1e0a7d7e9de83e => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:10.489316 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d5cb6-45fe-43bb-83fa-df63004225ad admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:10.490698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-734d5cb6-45fe-43bb-83fa-df63004225ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:10.504029 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d5cb6-45fe-43bb-83fa-df63004225ad admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:10.564100 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb90b4-3acd-4ebf-8ff9-be40a193c867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:10.564734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb90b4-3acd-4ebf-8ff9-be40a193c867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:10.565020 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb90b4-3acd-4ebf-8ff9-be40a193c867 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:10.565299 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb90b4-3acd-4ebf-8ff9-be40a193c867 admin admin] PATH_INFO: `/v3/projects/54499775af2b42368144250c5997496f` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:10.566694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb90b4-3acd-4ebf-8ff9-be40a193c867 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54499775af2b42368144250c5997496f)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:10.568646 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb90b4-3acd-4ebf-8ff9-be40a193c867 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:10.614449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-06bb90b4-3acd-4ebf-8ff9-be40a193c867 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54499775af2b42368144250c5997496f', 'request_id': 'req-06bb90b4-3acd-4ebf-8ff9-be40a193c867'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:10.621186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-06bb90b4-3acd-4ebf-8ff9-be40a193c867 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54499775af2b42368144250c5997496f', 'request_id': 'req-06bb90b4-3acd-4ebf-8ff9-be40a193c867'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:10.627968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-06bb90b4-3acd-4ebf-8ff9-be40a193c867 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54499775af2b42368144250c5997496f', 'request_id': 'req-06bb90b4-3acd-4ebf-8ff9-be40a193c867'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:10.638154 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-06bb90b4-3acd-4ebf-8ff9-be40a193c867 admin admin] The token cache is being invalidate because project 54499775af2b42368144250c5997496f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:51:10.638435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-06bb90b4-3acd-4ebf-8ff9-be40a193c867 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:10.639093 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-06bb90b4-3acd-4ebf-8ff9-be40a193c867 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:10.641512 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 955/1930] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:51:10 2022] DELETE /identity/v3/projects/54499775af2b42368144250c5997496f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:51:10.647676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081b8ab-48f1-47bb-bc4d-1c9070982671 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:10.648189 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081b8ab-48f1-47bb-bc4d-1c9070982671 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:10.648285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081b8ab-48f1-47bb-bc4d-1c9070982671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:10.667724 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-4081b8ab-48f1-47bb-bc4d-1c9070982671 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:10.669230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4081b8ab-48f1-47bb-bc4d-1c9070982671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:10.727421 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 976/1931] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:10.736127 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ea41c-562e-4607-8f47-fa9b80b2e4fd None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:10.736620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ea41c-562e-4607-8f47-fa9b80b2e4fd None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:10.737034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ea41c-562e-4607-8f47-fa9b80b2e4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:10.763217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-336ea41c-562e-4607-8f47-fa9b80b2e4fd None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:10.764987 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-336ea41c-562e-4607-8f47-fa9b80b2e4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:10.772870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 956/1932] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:10.777203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081b8ab-48f1-47bb-bc4d-1c9070982671 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:10.787042 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081b8ab-48f1-47bb-bc4d-1c9070982671 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:10.791820 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1160c9-db87-4b8d-8620-48d34171b918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:10.792451 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1160c9-db87-4b8d-8620-48d34171b918 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:10.792757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1160c9-db87-4b8d-8620-48d34171b918 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:10.793029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1160c9-db87-4b8d-8620-48d34171b918 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:10.793702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1160c9-db87-4b8d-8620-48d34171b918 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:51:10.794006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1160c9-db87-4b8d-8620-48d34171b918 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:10.795722 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1160c9-db87-4b8d-8620-48d34171b918 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:10.807729 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 977/1933] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:51:10.812864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ea41c-562e-4607-8f47-fa9b80b2e4fd None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:10.822132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ea41c-562e-4607-8f47-fa9b80b2e4fd None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:10.830111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74481b-d117-4596-ad30-f1a806f8bd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:10.830722 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74481b-d117-4596-ad30-f1a806f8bd77 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:10.830998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74481b-d117-4596-ad30-f1a806f8bd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:10.831293 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74481b-d117-4596-ad30-f1a806f8bd77 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:10.832015 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a74481b-d117-4596-ad30-f1a806f8bd77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:10.834052 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a74481b-d117-4596-ad30-f1a806f8bd77 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:10.867191 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1160c9-db87-4b8d-8620-48d34171b918 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:10.872358 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 957/1934] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:51:10 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:51:10.878967 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1160c9-db87-4b8d-8620-48d34171b918 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:10.884036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:10.959015 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e983cf9-aae8-49b5-8b89-5d5aa129dbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:10.959931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e983cf9-aae8-49b5-8b89-5d5aa129dbe6 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:10.960449 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e983cf9-aae8-49b5-8b89-5d5aa129dbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:10.960750 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e983cf9-aae8-49b5-8b89-5d5aa129dbe6 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:10.962338 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e983cf9-aae8-49b5-8b89-5d5aa129dbe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:10.964858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e983cf9-aae8-49b5-8b89-5d5aa129dbe6 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:10.992189 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 978/1935] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:51:10 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:51:10.996404 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74481b-d117-4596-ad30-f1a806f8bd77 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:11.007689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74481b-d117-4596-ad30-f1a806f8bd77 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:11.016313 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e00866-6bad-4f5f-ad1c-d527fcd4bb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:11.016313 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e00866-6bad-4f5f-ad1c-d527fcd4bb21 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:11.016313 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e00866-6bad-4f5f-ad1c-d527fcd4bb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:11.016313 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e00866-6bad-4f5f-ad1c-d527fcd4bb21 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:11.016980 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e00866-6bad-4f5f-ad1c-d527fcd4bb21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:11.029315 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e00866-6bad-4f5f-ad1c-d527fcd4bb21 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:11.041291 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:11.052181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 958/1936] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:51:11.066371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e983cf9-aae8-49b5-8b89-5d5aa129dbe6 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:11.080381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e983cf9-aae8-49b5-8b89-5d5aa129dbe6 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:11.085470 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679de598-3de5-490e-a22b-fdcdbddc4613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:11.086219 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679de598-3de5-490e-a22b-fdcdbddc4613 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:11.086596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679de598-3de5-490e-a22b-fdcdbddc4613 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:11.086968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679de598-3de5-490e-a22b-fdcdbddc4613 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:11.087710 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679de598-3de5-490e-a22b-fdcdbddc4613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:11.089969 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679de598-3de5-490e-a22b-fdcdbddc4613 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:11.113839 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 979/1937] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:51:11.118441 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e00866-6bad-4f5f-ad1c-d527fcd4bb21 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:11.137616 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e00866-6bad-4f5f-ad1c-d527fcd4bb21 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:11.151305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc931497-ee67-41ca-98df-f75a398b72d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:11.153158 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc931497-ee67-41ca-98df-f75a398b72d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:11.153657 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc931497-ee67-41ca-98df-f75a398b72d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:11.154436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc931497-ee67-41ca-98df-f75a398b72d3 admin admin] PATH_INFO: `/v3/projects/c48ec3b088624fcdbbd2c4bf67e47613/users/9b409ca45ec74c04808a668af74e1bcb/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:11.175520 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc931497-ee67-41ca-98df-f75a398b72d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c48ec3b088624fcdbbd2c4bf67e47613, user_id=9b409ca45ec74c04808a668af74e1bcb, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:11.178116 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc931497-ee67-41ca-98df-f75a398b72d3 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:11.210709 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 959/1938] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:51:11 2022] PUT /identity/v3/projects/c48ec3b088624fcdbbd2c4bf67e47613/users/9b409ca45ec74c04808a668af74e1bcb/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:11.224554 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac87b5-4b72-472c-ad5a-1803c7b56903 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:11.225096 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac87b5-4b72-472c-ad5a-1803c7b56903 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:11.225545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac87b5-4b72-472c-ad5a-1803c7b56903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:11.262838 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-6eac87b5-4b72-472c-ad5a-1803c7b56903 None None] MFA Rules not processed for user `9b409ca45ec74c04808a668af74e1bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:11.265166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-6eac87b5-4b72-472c-ad5a-1803c7b56903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:11.349199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 980/1939] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:11.362501 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc931497-ee67-41ca-98df-f75a398b72d3 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:11.364401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-fc931497-ee67-41ca-98df-f75a398b72d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:11.428243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc931497-ee67-41ca-98df-f75a398b72d3 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:11.432262 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720f383-046e-4403-b832-ee212ee4f99d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:11.432913 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720f383-046e-4403-b832-ee212ee4f99d service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:11.433195 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720f383-046e-4403-b832-ee212ee4f99d service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:11.433473 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720f383-046e-4403-b832-ee212ee4f99d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:11.453859 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720f383-046e-4403-b832-ee212ee4f99d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:11.456456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720f383-046e-4403-b832-ee212ee4f99d service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:11.477028 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 960/1940] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:51:21.346748 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:21.462238 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=36,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:26.916780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac87b5-4b72-472c-ad5a-1803c7b56903 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:26.918228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-6eac87b5-4b72-472c-ad5a-1803c7b56903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:26.986696 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac87b5-4b72-472c-ad5a-1803c7b56903 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:26.991535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd4ffd-7096-45a6-aeff-71e30d55cac3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:26.992162 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd4ffd-7096-45a6-aeff-71e30d55cac3 service placement] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:26.992444 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd4ffd-7096-45a6-aeff-71e30d55cac3 service placement] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:26.992716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd4ffd-7096-45a6-aeff-71e30d55cac3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:26.994409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-00cd4ffd-7096-45a6-aeff-71e30d55cac3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:27.005382 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd4ffd-7096-45a6-aeff-71e30d55cac3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:27.006940 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd4ffd-7096-45a6-aeff-71e30d55cac3 service placement] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:27.017071 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 981/1941] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:51:35.475726 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720f383-046e-4403-b832-ee212ee4f99d service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:35.544953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720f383-046e-4403-b832-ee212ee4f99d service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:35.550357 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c6d91-152d-49d9-a25d-2dce483307c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:35.551016 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c6d91-152d-49d9-a25d-2dce483307c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:35.551291 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c6d91-152d-49d9-a25d-2dce483307c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:35.551561 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c6d91-152d-49d9-a25d-2dce483307c0 admin admin] PATH_INFO: `/v3/users/9b409ca45ec74c04808a668af74e1bcb` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:35.552683 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c6d91-152d-49d9-a25d-2dce483307c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b409ca45ec74c04808a668af74e1bcb)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:35.554596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c6d91-152d-49d9-a25d-2dce483307c0 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:35.609378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2d4c6d91-152d-49d9-a25d-2dce483307c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b409ca45ec74c04808a668af74e1bcb'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:35.609721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2d4c6d91-152d-49d9-a25d-2dce483307c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b409ca45ec74c04808a668af74e1bcb'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:35.622988 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2d4c6d91-152d-49d9-a25d-2dce483307c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b409ca45ec74c04808a668af74e1bcb'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:35.629945 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2d4c6d91-152d-49d9-a25d-2dce483307c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b409ca45ec74c04808a668af74e1bcb'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:35.630653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2d4c6d91-152d-49d9-a25d-2dce483307c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b409ca45ec74c04808a668af74e1bcb'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:35.643312 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 961/1942] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:51:35 2022] DELETE /identity/v3/users/9b409ca45ec74c04808a668af74e1bcb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:35.647794 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f68c04-903c-4073-8ec0-d6c8ef07b670 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:35.648286 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f68c04-903c-4073-8ec0-d6c8ef07b670 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:35.648389 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f68c04-903c-4073-8ec0-d6c8ef07b670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:35.668272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-54f68c04-903c-4073-8ec0-d6c8ef07b670 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:35.669478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-54f68c04-903c-4073-8ec0-d6c8ef07b670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:35.735785 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 982/1943] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:35.742756 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c6d91-152d-49d9-a25d-2dce483307c0 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:35.744246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-2d4c6d91-152d-49d9-a25d-2dce483307c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:35.822692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c6d91-152d-49d9-a25d-2dce483307c0 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:35.831115 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c0df3-2616-4569-b526-a6b03907585d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:35.831798 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c0df3-2616-4569-b526-a6b03907585d service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:35.832105 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c0df3-2616-4569-b526-a6b03907585d service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:35.832389 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c0df3-2616-4569-b526-a6b03907585d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:35.843918 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c0df3-2616-4569-b526-a6b03907585d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:35.845746 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c0df3-2616-4569-b526-a6b03907585d service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:35.856068 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 962/1944] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:51:36.393714 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9813dfb-ede4-4ad0-8cd9-058eaff19ba6 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:36.394150 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9813dfb-ede4-4ad0-8cd9-058eaff19ba6 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:36.395235 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9813dfb-ede4-4ad0-8cd9-058eaff19ba6 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:36.397045 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 983/1945] 10.222.0.36 () {54 vars in 921 bytes} [Sun Oct 30 00:51:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:51:36.425129 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263abff-ba02-476b-bbca-40b7c322bfe7 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:36.425926 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263abff-ba02-476b-bbca-40b7c322bfe7 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:36.426225 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263abff-ba02-476b-bbca-40b7c322bfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:36.455806 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-5263abff-ba02-476b-bbca-40b7c322bfe7 None None] MFA Rules not processed for user `5ded89d48d6c4a15a5c5829f594f8776`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:36.457182 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-5263abff-ba02-476b-bbca-40b7c322bfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:36.483682 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:36.547711 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 963/1946] 10.222.0.36 () {58 vars in 1018 bytes} [Sun Oct 30 00:51:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:36.674944 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9813dfb-ede4-4ad0-8cd9-058eaff19ba6 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:36.676773 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-b9813dfb-ede4-4ad0-8cd9-058eaff19ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:36.690755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263abff-ba02-476b-bbca-40b7c322bfe7 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:36.694317 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-5263abff-ba02-476b-bbca-40b7c322bfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:36.701526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9813dfb-ede4-4ad0-8cd9-058eaff19ba6 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:36.709829 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263abff-ba02-476b-bbca-40b7c322bfe7 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:36.711185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b74526-8314-4d20-96de-e4f962687028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:36.712194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b74526-8314-4d20-96de-e4f962687028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:36.712661 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b74526-8314-4d20-96de-e4f962687028 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:36.713137 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b74526-8314-4d20-96de-e4f962687028 admin admin] PATH_INFO: `/v3/projects/c48ec3b088624fcdbbd2c4bf67e47613` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:36.714736 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b74526-8314-4d20-96de-e4f962687028 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c48ec3b088624fcdbbd2c4bf67e47613)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:36.716297 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20f65e-3aa3-4a07-9c01-c30d5c3edd3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:36.716913 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20f65e-3aa3-4a07-9c01-c30d5c3edd3b service placement] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:36.717191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20f65e-3aa3-4a07-9c01-c30d5c3edd3b service placement] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:36.717504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20f65e-3aa3-4a07-9c01-c30d5c3edd3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:36.718314 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b74526-8314-4d20-96de-e4f962687028 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:36.738748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20f65e-3aa3-4a07-9c01-c30d5c3edd3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:36.741708 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20f65e-3aa3-4a07-9c01-c30d5c3edd3b service placement] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:36.764215 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 964/1947] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:51:36.799310 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a2b74526-8314-4d20-96de-e4f962687028 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c48ec3b088624fcdbbd2c4bf67e47613', 'request_id': 'req-a2b74526-8314-4d20-96de-e4f962687028'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:36.813711 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a2b74526-8314-4d20-96de-e4f962687028 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c48ec3b088624fcdbbd2c4bf67e47613', 'request_id': 'req-a2b74526-8314-4d20-96de-e4f962687028'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:36.820690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a2b74526-8314-4d20-96de-e4f962687028 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c48ec3b088624fcdbbd2c4bf67e47613', 'request_id': 'req-a2b74526-8314-4d20-96de-e4f962687028'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:36.833512 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a2b74526-8314-4d20-96de-e4f962687028 admin admin] The token cache is being invalidate because project c48ec3b088624fcdbbd2c4bf67e47613 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:51:36.833935 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a2b74526-8314-4d20-96de-e4f962687028 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:36.834697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a2b74526-8314-4d20-96de-e4f962687028 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:36.836784 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 984/1948] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:51:36 2022] DELETE /identity/v3/projects/c48ec3b088624fcdbbd2c4bf67e47613 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:36.842274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd9de5-6057-49c4-a939-5c53e1f2da68 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:36.844010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd9de5-6057-49c4-a939-5c53e1f2da68 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:36.845504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd9de5-6057-49c4-a939-5c53e1f2da68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:36.881642 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-79bd9de5-6057-49c4-a939-5c53e1f2da68 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:36.882867 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-79bd9de5-6057-49c4-a939-5c53e1f2da68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:36.958888 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 965/1949] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:36.967937 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ba727-c25b-4ce4-9760-7780e7530d2f None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:36.968495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ba727-c25b-4ce4-9760-7780e7530d2f None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:36.968849 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ba727-c25b-4ce4-9760-7780e7530d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:37.000017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-f78ba727-c25b-4ce4-9760-7780e7530d2f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:37.002406 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f78ba727-c25b-4ce4-9760-7780e7530d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:37.015664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 985/1950] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:37.017959 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd9de5-6057-49c4-a939-5c53e1f2da68 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:37.036201 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd9de5-6057-49c4-a939-5c53e1f2da68 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:37.041762 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994810e-8150-4d0d-b72c-b928108f17a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:37.042397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994810e-8150-4d0d-b72c-b928108f17a9 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:37.042680 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994810e-8150-4d0d-b72c-b928108f17a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:37.042955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994810e-8150-4d0d-b72c-b928108f17a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:37.043781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994810e-8150-4d0d-b72c-b928108f17a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:51:37.044154 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994810e-8150-4d0d-b72c-b928108f17a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:37.047247 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994810e-8150-4d0d-b72c-b928108f17a9 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:37.059866 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 966/1951] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:51:37.064465 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ba727-c25b-4ce4-9760-7780e7530d2f None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:37.074366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ba727-c25b-4ce4-9760-7780e7530d2f None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:37.080374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109db2d-76c7-4dbd-8e78-bce304bd49ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:37.081026 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109db2d-76c7-4dbd-8e78-bce304bd49ef admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:37.081306 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109db2d-76c7-4dbd-8e78-bce304bd49ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:37.081570 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109db2d-76c7-4dbd-8e78-bce304bd49ef admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:37.082211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109db2d-76c7-4dbd-8e78-bce304bd49ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:37.084155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109db2d-76c7-4dbd-8e78-bce304bd49ef admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:37.089225 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:37.117292 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 986/1952] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:51:37 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:51:37.121715 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994810e-8150-4d0d-b72c-b928108f17a9 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:37.133989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994810e-8150-4d0d-b72c-b928108f17a9 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:37.205475 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6122060-4722-4af9-b7b8-828efeeb34d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:37.206105 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6122060-4722-4af9-b7b8-828efeeb34d0 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:37.206382 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6122060-4722-4af9-b7b8-828efeeb34d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:37.206662 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6122060-4722-4af9-b7b8-828efeeb34d0 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:37.207459 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6122060-4722-4af9-b7b8-828efeeb34d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:37.209509 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6122060-4722-4af9-b7b8-828efeeb34d0 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:37.234594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 967/1953] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:51:37 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:51:37.238706 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109db2d-76c7-4dbd-8e78-bce304bd49ef admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:37.252893 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109db2d-76c7-4dbd-8e78-bce304bd49ef admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:37.259015 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c538a2-7c11-45e3-9e17-4ea2c7bd915c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:37.259702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c538a2-7c11-45e3-9e17-4ea2c7bd915c admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:37.260050 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c538a2-7c11-45e3-9e17-4ea2c7bd915c admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:37.260433 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c538a2-7c11-45e3-9e17-4ea2c7bd915c admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:37.261156 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c538a2-7c11-45e3-9e17-4ea2c7bd915c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:37.263829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c538a2-7c11-45e3-9e17-4ea2c7bd915c admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:37.284841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 987/1954] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:51:37.286060 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6122060-4722-4af9-b7b8-828efeeb34d0 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:37.297703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6122060-4722-4af9-b7b8-828efeeb34d0 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:37.303383 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16fcc8-ba82-44f7-98e2-73c30630a55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:37.304550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16fcc8-ba82-44f7-98e2-73c30630a55c admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:37.304550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16fcc8-ba82-44f7-98e2-73c30630a55c admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:37.304550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16fcc8-ba82-44f7-98e2-73c30630a55c admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:37.305195 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16fcc8-ba82-44f7-98e2-73c30630a55c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:37.306809 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16fcc8-ba82-44f7-98e2-73c30630a55c admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:37.322525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 968/1955] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:51:37.323708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c538a2-7c11-45e3-9e17-4ea2c7bd915c admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:37.334473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c538a2-7c11-45e3-9e17-4ea2c7bd915c admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:37.340299 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cfe1b2-5813-424d-94d9-f7cc581320c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:37.340850 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfe1b2-5813-424d-94d9-f7cc581320c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:37.341128 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfe1b2-5813-424d-94d9-f7cc581320c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:37.341390 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfe1b2-5813-424d-94d9-f7cc581320c8 admin admin] PATH_INFO: `/v3/projects/feb686356f3244e9b69153cda8953c99/users/55e6be4152254f668805883f56e6da53/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:37.363469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16fcc8-ba82-44f7-98e2-73c30630a55c admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:37.365325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-0e16fcc8-ba82-44f7-98e2-73c30630a55c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:37.368199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cfe1b2-5813-424d-94d9-f7cc581320c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=feb686356f3244e9b69153cda8953c99, user_id=55e6be4152254f668805883f56e6da53, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:37.377518 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cfe1b2-5813-424d-94d9-f7cc581320c8 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:37.379702 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16fcc8-ba82-44f7-98e2-73c30630a55c admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:37.385045 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a968f3f-b196-4d62-ac72-cbdf23de0cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:37.385624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a968f3f-b196-4d62-ac72-cbdf23de0cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:37.385907 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a968f3f-b196-4d62-ac72-cbdf23de0cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:37.386183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a968f3f-b196-4d62-ac72-cbdf23de0cd5 admin admin] PATH_INFO: `/v3/users/8a99aa8eafb4487ab539a3fa8199f626` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:37.387447 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a968f3f-b196-4d62-ac72-cbdf23de0cd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a99aa8eafb4487ab539a3fa8199f626)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:37.389441 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a968f3f-b196-4d62-ac72-cbdf23de0cd5 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:37.394759 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:37.400957 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 988/1956] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:51:37 2022] PUT /identity/v3/projects/feb686356f3244e9b69153cda8953c99/users/55e6be4152254f668805883f56e6da53/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:37.413099 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c7af5-4df9-41d3-be66-fac39d458497 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:37.413466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c7af5-4df9-41d3-be66-fac39d458497 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:37.413838 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c7af5-4df9-41d3-be66-fac39d458497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:37.455428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-184c7af5-4df9-41d3-be66-fac39d458497 None None] MFA Rules not processed for user `55e6be4152254f668805883f56e6da53`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:37.457033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-184c7af5-4df9-41d3-be66-fac39d458497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:37.459583 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2a968f3f-b196-4d62-ac72-cbdf23de0cd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a99aa8eafb4487ab539a3fa8199f626'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:37.460601 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2a968f3f-b196-4d62-ac72-cbdf23de0cd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a99aa8eafb4487ab539a3fa8199f626'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:37.478613 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2a968f3f-b196-4d62-ac72-cbdf23de0cd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a99aa8eafb4487ab539a3fa8199f626'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:37.488973 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2a968f3f-b196-4d62-ac72-cbdf23de0cd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a99aa8eafb4487ab539a3fa8199f626'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:37.489915 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2a968f3f-b196-4d62-ac72-cbdf23de0cd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a99aa8eafb4487ab539a3fa8199f626'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:37.508440 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 969/1957] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:51:37 2022] DELETE /identity/v3/users/8a99aa8eafb4487ab539a3fa8199f626 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:37.549318 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 989/1958] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:37.560991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a968f3f-b196-4d62-ac72-cbdf23de0cd5 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:37.563215 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-2a968f3f-b196-4d62-ac72-cbdf23de0cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:37.629593 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a968f3f-b196-4d62-ac72-cbdf23de0cd5 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:37.633429 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4177cd7-a672-478d-9aa0-f87584da2280 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:37.634047 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4177cd7-a672-478d-9aa0-f87584da2280 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:37.634325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4177cd7-a672-478d-9aa0-f87584da2280 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:37.634597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4177cd7-a672-478d-9aa0-f87584da2280 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:37.713233 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4177cd7-a672-478d-9aa0-f87584da2280 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:37.714706 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4177cd7-a672-478d-9aa0-f87584da2280 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:37.736226 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 970/1959] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:51:37.780678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c7af5-4df9-41d3-be66-fac39d458497 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:37.780678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-184c7af5-4df9-41d3-be66-fac39d458497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:37.784292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8ccad-440f-44ab-8c69-4d2583a55186 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:37.784582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8ccad-440f-44ab-8c69-4d2583a55186 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:37.784860 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8ccad-440f-44ab-8c69-4d2583a55186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:37.806942 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-e2b8ccad-440f-44ab-8c69-4d2583a55186 None None] MFA Rules not processed for user `350f5e4238f448949b2c372aabbfc835`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:37.808619 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e2b8ccad-440f-44ab-8c69-4d2583a55186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:37.856460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:37.865269 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c7af5-4df9-41d3-be66-fac39d458497 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:37.896255 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 971/1960] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:37.924838 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:37.944710 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e20dd6-62da-43ee-be47-0d593aa30487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:37.945847 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e20dd6-62da-43ee-be47-0d593aa30487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:37.946729 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e20dd6-62da-43ee-be47-0d593aa30487 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:37.947216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e20dd6-62da-43ee-be47-0d593aa30487 admin admin] PATH_INFO: `/v3/projects/2d6a4e760923417b958e27365564dc17` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:37.949477 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e20dd6-62da-43ee-be47-0d593aa30487 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d6a4e760923417b958e27365564dc17)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:37.951538 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e20dd6-62da-43ee-be47-0d593aa30487 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:38.013537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-c8e20dd6-62da-43ee-be47-0d593aa30487 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d6a4e760923417b958e27365564dc17', 'request_id': 'req-c8e20dd6-62da-43ee-be47-0d593aa30487'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.024119 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-c8e20dd6-62da-43ee-be47-0d593aa30487 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d6a4e760923417b958e27365564dc17', 'request_id': 'req-c8e20dd6-62da-43ee-be47-0d593aa30487'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.034778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-c8e20dd6-62da-43ee-be47-0d593aa30487 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d6a4e760923417b958e27365564dc17', 'request_id': 'req-c8e20dd6-62da-43ee-be47-0d593aa30487'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.051672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-c8e20dd6-62da-43ee-be47-0d593aa30487 admin admin] The token cache is being invalidate because project 2d6a4e760923417b958e27365564dc17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:51:38.052049 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-c8e20dd6-62da-43ee-be47-0d593aa30487 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.052717 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-c8e20dd6-62da-43ee-be47-0d593aa30487 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.054229 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 990/1961] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:51:37 2022] DELETE /identity/v3/projects/2d6a4e760923417b958e27365564dc17 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:38.059600 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8ccad-440f-44ab-8c69-4d2583a55186 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:38.074005 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8ccad-440f-44ab-8c69-4d2583a55186 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:38.146950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11698598-02f6-45b0-852d-f620654901d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:38.147772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11698598-02f6-45b0-852d-f620654901d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:38.148062 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11698598-02f6-45b0-852d-f620654901d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:38.148372 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11698598-02f6-45b0-852d-f620654901d3 admin admin] PATH_INFO: `/v3/users/aa9603c7063146c0bbcbfe7b9d233857` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:38.149644 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11698598-02f6-45b0-852d-f620654901d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa9603c7063146c0bbcbfe7b9d233857)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:38.151967 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11698598-02f6-45b0-852d-f620654901d3 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:38.209857 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-11698598-02f6-45b0-852d-f620654901d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa9603c7063146c0bbcbfe7b9d233857'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.210608 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-11698598-02f6-45b0-852d-f620654901d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa9603c7063146c0bbcbfe7b9d233857'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.225980 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-11698598-02f6-45b0-852d-f620654901d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa9603c7063146c0bbcbfe7b9d233857'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.232896 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-11698598-02f6-45b0-852d-f620654901d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa9603c7063146c0bbcbfe7b9d233857'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.233701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-11698598-02f6-45b0-852d-f620654901d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa9603c7063146c0bbcbfe7b9d233857'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.248532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 972/1962] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:51:38 2022] DELETE /identity/v3/users/aa9603c7063146c0bbcbfe7b9d233857 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:38.421496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e20dd6-62da-43ee-be47-0d593aa30487 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:38.423350 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c8e20dd6-62da-43ee-be47-0d593aa30487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:38.437402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e20dd6-62da-43ee-be47-0d593aa30487 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:38.502344 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ff4ff-388d-44f8-b0b4-802963862887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:38.502973 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ff4ff-388d-44f8-b0b4-802963862887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:38.503272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ff4ff-388d-44f8-b0b4-802963862887 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:38.503528 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ff4ff-388d-44f8-b0b4-802963862887 admin admin] PATH_INFO: `/v3/projects/594b7a467b694173b87556ff59e3892a` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:38.504676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ff4ff-388d-44f8-b0b4-802963862887 admin admin] RBAC: Authorizing `identity:delete_project(project_id=594b7a467b694173b87556ff59e3892a)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:38.506582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ff4ff-388d-44f8-b0b4-802963862887 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:38.555226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-455ff4ff-388d-44f8-b0b4-802963862887 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '594b7a467b694173b87556ff59e3892a', 'request_id': 'req-455ff4ff-388d-44f8-b0b4-802963862887'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.566318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11698598-02f6-45b0-852d-f620654901d3 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:38.567053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-455ff4ff-388d-44f8-b0b4-802963862887 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '594b7a467b694173b87556ff59e3892a', 'request_id': 'req-455ff4ff-388d-44f8-b0b4-802963862887'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.568430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-11698598-02f6-45b0-852d-f620654901d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:38.572931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-455ff4ff-388d-44f8-b0b4-802963862887 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '594b7a467b694173b87556ff59e3892a', 'request_id': 'req-455ff4ff-388d-44f8-b0b4-802963862887'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.581243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11698598-02f6-45b0-852d-f620654901d3 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:38.585707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-455ff4ff-388d-44f8-b0b4-802963862887 admin admin] The token cache is being invalidate because project 594b7a467b694173b87556ff59e3892a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:51:38.586102 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-455ff4ff-388d-44f8-b0b4-802963862887 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.586769 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-455ff4ff-388d-44f8-b0b4-802963862887 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.588372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 991/1963] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:51:38 2022] DELETE /identity/v3/projects/594b7a467b694173b87556ff59e3892a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:38.593430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a63537-3145-4f9d-8186-d4592bbd72d2 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:38.593789 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a63537-3145-4f9d-8186-d4592bbd72d2 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:38.594301 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a63537-3145-4f9d-8186-d4592bbd72d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:38.608036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:38.614051 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-c3a63537-3145-4f9d-8186-d4592bbd72d2 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:38.615681 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c3a63537-3145-4f9d-8186-d4592bbd72d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:38.645694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1bd733-13b6-458e-9aaa-bba1f8ed91e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:38.646427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bd733-13b6-458e-9aaa-bba1f8ed91e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:38.646715 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bd733-13b6-458e-9aaa-bba1f8ed91e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:38.647041 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bd733-13b6-458e-9aaa-bba1f8ed91e8 admin admin] PATH_INFO: `/v3/users/abfafb58fd6f4b1eb3a904af79f39723` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:38.648202 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1bd733-13b6-458e-9aaa-bba1f8ed91e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abfafb58fd6f4b1eb3a904af79f39723)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:38.650134 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1bd733-13b6-458e-9aaa-bba1f8ed91e8 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:38.652457 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:38.678845 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 992/1964] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:51:38.687165 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27388c-6bcb-4696-8461-ec6cedceae85 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:38.687415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27388c-6bcb-4696-8461-ec6cedceae85 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:38.687677 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27388c-6bcb-4696-8461-ec6cedceae85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:38.700710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8c1bd733-13b6-458e-9aaa-bba1f8ed91e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abfafb58fd6f4b1eb3a904af79f39723'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.702250 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8c1bd733-13b6-458e-9aaa-bba1f8ed91e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abfafb58fd6f4b1eb3a904af79f39723'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.708260 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-6e27388c-6bcb-4696-8461-ec6cedceae85 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:38.709467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-6e27388c-6bcb-4696-8461-ec6cedceae85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:38.713742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8c1bd733-13b6-458e-9aaa-bba1f8ed91e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abfafb58fd6f4b1eb3a904af79f39723'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.721769 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 993/1965] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:51:38.721922 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8c1bd733-13b6-458e-9aaa-bba1f8ed91e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abfafb58fd6f4b1eb3a904af79f39723'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.722461 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8c1bd733-13b6-458e-9aaa-bba1f8ed91e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abfafb58fd6f4b1eb3a904af79f39723'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:38.722911 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27388c-6bcb-4696-8461-ec6cedceae85 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:38.732453 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27388c-6bcb-4696-8461-ec6cedceae85 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:38.736355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 973/1966] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:51:38 2022] DELETE /identity/v3/users/abfafb58fd6f4b1eb3a904af79f39723 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:38.737218 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b5529-9017-47c9-bd33-91c2bac77faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:38.737817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b5529-9017-47c9-bd33-91c2bac77faa admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:38.738098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b5529-9017-47c9-bd33-91c2bac77faa admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:38.738370 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b5529-9017-47c9-bd33-91c2bac77faa admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:38.739018 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b5529-9017-47c9-bd33-91c2bac77faa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:51:38.739320 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b5529-9017-47c9-bd33-91c2bac77faa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:38.740790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6397ac-4583-4865-8f77-e7b1088ea935 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:38.740949 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b5529-9017-47c9-bd33-91c2bac77faa admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:38.741066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6397ac-4583-4865-8f77-e7b1088ea935 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:38.741353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6397ac-4583-4865-8f77-e7b1088ea935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:38.756901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 994/1967] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:51:38.757950 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b5529-9017-47c9-bd33-91c2bac77faa admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:38.760876 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-4c6397ac-4583-4865-8f77-e7b1088ea935 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:38.762273 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4c6397ac-4583-4865-8f77-e7b1088ea935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:38.768319 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b5529-9017-47c9-bd33-91c2bac77faa admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:38.822795 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 974/1968] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:38.828483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5bfe2-d50c-44e9-9adb-3e636ae76eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:38.829133 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5bfe2-d50c-44e9-9adb-3e636ae76eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:38.829415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5bfe2-d50c-44e9-9adb-3e636ae76eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:38.829690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5bfe2-d50c-44e9-9adb-3e636ae76eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:38.830354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5bfe2-d50c-44e9-9adb-3e636ae76eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:38.832326 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5bfe2-d50c-44e9-9adb-3e636ae76eb7 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:38.833856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6397ac-4583-4865-8f77-e7b1088ea935 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:38.835352 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4c6397ac-4583-4865-8f77-e7b1088ea935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:38.880953 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 995/1969] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:51:38 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:51:38.881906 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5bfe2-d50c-44e9-9adb-3e636ae76eb7 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:38.912326 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6397ac-4583-4865-8f77-e7b1088ea935 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:38.948858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:38.960786 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5bfe2-d50c-44e9-9adb-3e636ae76eb7 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:38.966434 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78a1f4-27af-4e3d-9057-0460bab9c7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:38.967063 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78a1f4-27af-4e3d-9057-0460bab9c7a4 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:38.967372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78a1f4-27af-4e3d-9057-0460bab9c7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:38.968278 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78a1f4-27af-4e3d-9057-0460bab9c7a4 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:38.968917 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78a1f4-27af-4e3d-9057-0460bab9c7a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:38.970873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78a1f4-27af-4e3d-9057-0460bab9c7a4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:38.973397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f23c9-2ca2-4654-8382-b07fa707dcb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:38.974012 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f23c9-2ca2-4654-8382-b07fa707dcb3 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:38.974296 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f23c9-2ca2-4654-8382-b07fa707dcb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:38.974582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f23c9-2ca2-4654-8382-b07fa707dcb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:38.985497 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f23c9-2ca2-4654-8382-b07fa707dcb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:38.987397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f23c9-2ca2-4654-8382-b07fa707dcb3 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:38.998116 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 975/1970] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:51:39.002353 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 996/1971] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:51:38 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:51:39.006349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f23c9-2ca2-4654-8382-b07fa707dcb3 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:39.016514 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f23c9-2ca2-4654-8382-b07fa707dcb3 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:39.021104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d53fcd-089d-4c49-ac34-955060bdf269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:39.021695 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53fcd-089d-4c49-ac34-955060bdf269 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:39.021972 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53fcd-089d-4c49-ac34-955060bdf269 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:39.022243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53fcd-089d-4c49-ac34-955060bdf269 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:39.022869 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d53fcd-089d-4c49-ac34-955060bdf269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:39.024542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d53fcd-089d-4c49-ac34-955060bdf269 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:39.035505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 976/1972] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:51:39.040154 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78a1f4-27af-4e3d-9057-0460bab9c7a4 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:39.045535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:39.052756 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78a1f4-27af-4e3d-9057-0460bab9c7a4 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:39.059595 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c772e2-5efe-4e90-a65f-86a4f2e2f1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:39.060458 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c772e2-5efe-4e90-a65f-86a4f2e2f1a4 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:39.060745 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c772e2-5efe-4e90-a65f-86a4f2e2f1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:39.061713 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c772e2-5efe-4e90-a65f-86a4f2e2f1a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:39.062350 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c772e2-5efe-4e90-a65f-86a4f2e2f1a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:39.064672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c772e2-5efe-4e90-a65f-86a4f2e2f1a4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:39.077651 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 997/1973] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:51:39.082554 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d53fcd-089d-4c49-ac34-955060bdf269 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:39.093371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d53fcd-089d-4c49-ac34-955060bdf269 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:39.098530 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8a26f-49ba-40e8-8f6f-c33fff65232b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:39.099423 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8a26f-49ba-40e8-8f6f-c33fff65232b admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:39.099874 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8a26f-49ba-40e8-8f6f-c33fff65232b admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:39.100321 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8a26f-49ba-40e8-8f6f-c33fff65232b admin admin] PATH_INFO: `/v3/projects/ff911bf108474b97871110ce775df4a2/users/dca2706e6dc0494096a0d8b14ddb13ec/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:39.116626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8a26f-49ba-40e8-8f6f-c33fff65232b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff911bf108474b97871110ce775df4a2, user_id=dca2706e6dc0494096a0d8b14ddb13ec, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:39.118963 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8a26f-49ba-40e8-8f6f-c33fff65232b admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:39.133650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 977/1974] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:51:39 2022] PUT /identity/v3/projects/ff911bf108474b97871110ce775df4a2/users/dca2706e6dc0494096a0d8b14ddb13ec/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:39.146175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367b7b2-7a09-4db9-9f58-ffefb1c40fb6 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:39.146548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367b7b2-7a09-4db9-9f58-ffefb1c40fb6 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:39.146913 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367b7b2-7a09-4db9-9f58-ffefb1c40fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:39.190055 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-8367b7b2-7a09-4db9-9f58-ffefb1c40fb6 None None] MFA Rules not processed for user `dca2706e6dc0494096a0d8b14ddb13ec`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:39.191294 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-8367b7b2-7a09-4db9-9f58-ffefb1c40fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:39.269996 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 998/1975] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:39.283095 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8a26f-49ba-40e8-8f6f-c33fff65232b admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:39.284993 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-03a8a26f-49ba-40e8-8f6f-c33fff65232b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:39.288263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367b7b2-7a09-4db9-9f58-ffefb1c40fb6 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:39.290154 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-8367b7b2-7a09-4db9-9f58-ffefb1c40fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:39.357462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8a26f-49ba-40e8-8f6f-c33fff65232b admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:39.362461 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af049c1-c979-426a-97dd-35752d7a5c35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:39.363168 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af049c1-c979-426a-97dd-35752d7a5c35 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:39.363451 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af049c1-c979-426a-97dd-35752d7a5c35 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:39.363720 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af049c1-c979-426a-97dd-35752d7a5c35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:39.372917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af049c1-c979-426a-97dd-35752d7a5c35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:39.374506 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af049c1-c979-426a-97dd-35752d7a5c35 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:39.378845 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367b7b2-7a09-4db9-9f58-ffefb1c40fb6 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:39.383652 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515bf7a3-2e68-452f-949f-19714121c7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:39.384527 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bf7a3-2e68-452f-949f-19714121c7ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:39.384651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 978/1976] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:51:39.384747 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bf7a3-2e68-452f-949f-19714121c7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:39.384927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bf7a3-2e68-452f-949f-19714121c7ae admin admin] PATH_INFO: `/v3/projects/fffcab8b7320471b82c1d0f8d11153ed` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:39.386122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515bf7a3-2e68-452f-949f-19714121c7ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=fffcab8b7320471b82c1d0f8d11153ed)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:39.388070 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515bf7a3-2e68-452f-949f-19714121c7ae admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:39.409582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af049c1-c979-426a-97dd-35752d7a5c35 service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:39.424034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af049c1-c979-426a-97dd-35752d7a5c35 service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:39.436625 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e55b7e-c04d-4fda-8363-6e00cc1206e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:39.437555 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e55b7e-c04d-4fda-8363-6e00cc1206e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:39.437992 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e55b7e-c04d-4fda-8363-6e00cc1206e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:39.438652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e55b7e-c04d-4fda-8363-6e00cc1206e7 admin admin] PATH_INFO: `/v3/users/dca2706e6dc0494096a0d8b14ddb13ec` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:39.440156 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e55b7e-c04d-4fda-8363-6e00cc1206e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dca2706e6dc0494096a0d8b14ddb13ec)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:39.442889 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e55b7e-c04d-4fda-8363-6e00cc1206e7 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:39.448978 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-515bf7a3-2e68-452f-949f-19714121c7ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fffcab8b7320471b82c1d0f8d11153ed', 'request_id': 'req-515bf7a3-2e68-452f-949f-19714121c7ae'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:39.459222 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-515bf7a3-2e68-452f-949f-19714121c7ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fffcab8b7320471b82c1d0f8d11153ed', 'request_id': 'req-515bf7a3-2e68-452f-949f-19714121c7ae'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:39.466865 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-515bf7a3-2e68-452f-949f-19714121c7ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fffcab8b7320471b82c1d0f8d11153ed', 'request_id': 'req-515bf7a3-2e68-452f-949f-19714121c7ae'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:39.481010 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-515bf7a3-2e68-452f-949f-19714121c7ae admin admin] The token cache is being invalidate because project fffcab8b7320471b82c1d0f8d11153ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:51:39.481298 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-515bf7a3-2e68-452f-949f-19714121c7ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:39.482244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-515bf7a3-2e68-452f-949f-19714121c7ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:39.484473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 999/1977] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:51:39 2022] DELETE /identity/v3/projects/fffcab8b7320471b82c1d0f8d11153ed => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:51:39.509037 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-20e55b7e-c04d-4fda-8363-6e00cc1206e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dca2706e6dc0494096a0d8b14ddb13ec'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:39.509738 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-20e55b7e-c04d-4fda-8363-6e00cc1206e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dca2706e6dc0494096a0d8b14ddb13ec'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:39.524616 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-20e55b7e-c04d-4fda-8363-6e00cc1206e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dca2706e6dc0494096a0d8b14ddb13ec'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:39.533004 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-20e55b7e-c04d-4fda-8363-6e00cc1206e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dca2706e6dc0494096a0d8b14ddb13ec'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:39.533772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-20e55b7e-c04d-4fda-8363-6e00cc1206e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dca2706e6dc0494096a0d8b14ddb13ec'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:39.552291 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 979/1978] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:51:39 2022] DELETE /identity/v3/users/dca2706e6dc0494096a0d8b14ddb13ec => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:51:39.556312 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc914a3-337f-4dc3-b60b-15eb2fdfd810 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:39.556602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc914a3-337f-4dc3-b60b-15eb2fdfd810 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:39.556882 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc914a3-337f-4dc3-b60b-15eb2fdfd810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:39.600063 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-adc914a3-337f-4dc3-b60b-15eb2fdfd810 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:39.601466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-adc914a3-337f-4dc3-b60b-15eb2fdfd810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:39.608103 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:39.676564 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1000/1979] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:39.690595 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e55b7e-c04d-4fda-8363-6e00cc1206e7 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:39.692351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-20e55b7e-c04d-4fda-8363-6e00cc1206e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:39.715012 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:39.753720 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e55b7e-c04d-4fda-8363-6e00cc1206e7 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:39.757342 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7381f4d-b7c2-453c-bd1b-feef5ebacbd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:39.757948 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7381f4d-b7c2-453c-bd1b-feef5ebacbd7 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:39.758224 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7381f4d-b7c2-453c-bd1b-feef5ebacbd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:39.758495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7381f4d-b7c2-453c-bd1b-feef5ebacbd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:39.768103 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7381f4d-b7c2-453c-bd1b-feef5ebacbd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:39.769553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7381f4d-b7c2-453c-bd1b-feef5ebacbd7 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:39.778764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 980/1980] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:51:40.048962 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc914a3-337f-4dc3-b60b-15eb2fdfd810 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:40.050588 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-adc914a3-337f-4dc3-b60b-15eb2fdfd810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:40.063482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc914a3-337f-4dc3-b60b-15eb2fdfd810 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:40.068613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654acd4-db38-439f-a02e-0c2b7bdd4714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:40.069334 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654acd4-db38-439f-a02e-0c2b7bdd4714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:40.069620 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654acd4-db38-439f-a02e-0c2b7bdd4714 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:40.069918 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654acd4-db38-439f-a02e-0c2b7bdd4714 admin admin] PATH_INFO: `/v3/projects/ff911bf108474b97871110ce775df4a2` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:40.071090 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654acd4-db38-439f-a02e-0c2b7bdd4714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff911bf108474b97871110ce775df4a2)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:40.073065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654acd4-db38-439f-a02e-0c2b7bdd4714 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:40.120155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3654acd4-db38-439f-a02e-0c2b7bdd4714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff911bf108474b97871110ce775df4a2', 'request_id': 'req-3654acd4-db38-439f-a02e-0c2b7bdd4714'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:40.127642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3654acd4-db38-439f-a02e-0c2b7bdd4714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff911bf108474b97871110ce775df4a2', 'request_id': 'req-3654acd4-db38-439f-a02e-0c2b7bdd4714'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:40.133143 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3654acd4-db38-439f-a02e-0c2b7bdd4714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff911bf108474b97871110ce775df4a2', 'request_id': 'req-3654acd4-db38-439f-a02e-0c2b7bdd4714'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:40.144927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3654acd4-db38-439f-a02e-0c2b7bdd4714 admin admin] The token cache is being invalidate because project ff911bf108474b97871110ce775df4a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:51:40.145210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3654acd4-db38-439f-a02e-0c2b7bdd4714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:40.145830 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3654acd4-db38-439f-a02e-0c2b7bdd4714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:51:40.147296 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1001/1981] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:51:40 2022] DELETE /identity/v3/projects/ff911bf108474b97871110ce775df4a2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:40.152412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c8d4e-909c-4c31-a79e-b9c41b1b1e3b None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:40.152694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c8d4e-909c-4c31-a79e-b9c41b1b1e3b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:40.152967 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c8d4e-909c-4c31-a79e-b9c41b1b1e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:40.173692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-4d2c8d4e-909c-4c31-a79e-b9c41b1b1e3b None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:40.175315 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4d2c8d4e-909c-4c31-a79e-b9c41b1b1e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:40.233771 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 981/1982] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:51:40.241398 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95b2df-ef89-4905-afd8-ec5333f309d6 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:40.241680 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95b2df-ef89-4905-afd8-ec5333f309d6 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:40.241951 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95b2df-ef89-4905-afd8-ec5333f309d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:40.262209 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-7d95b2df-ef89-4905-afd8-ec5333f309d6 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:40.263399 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-7d95b2df-ef89-4905-afd8-ec5333f309d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:40.271610 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1002/1983] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:40.275719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c8d4e-909c-4c31-a79e-b9c41b1b1e3b None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:40.285423 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c8d4e-909c-4c31-a79e-b9c41b1b1e3b None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:40.289882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1a1d5-1df3-4ad6-adff-44d5c39b4987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:40.290474 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1a1d5-1df3-4ad6-adff-44d5c39b4987 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:40.290751 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1a1d5-1df3-4ad6-adff-44d5c39b4987 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:40.291023 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1a1d5-1df3-4ad6-adff-44d5c39b4987 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:40.291656 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db1a1d5-1df3-4ad6-adff-44d5c39b4987 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:51:40.291978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db1a1d5-1df3-4ad6-adff-44d5c39b4987 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:40.293628 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db1a1d5-1df3-4ad6-adff-44d5c39b4987 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:40.305456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 982/1984] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:51:40.309567 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95b2df-ef89-4905-afd8-ec5333f309d6 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:40.319583 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95b2df-ef89-4905-afd8-ec5333f309d6 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:40.324209 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4446a-39df-4b86-a4f6-b4b0ed174314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:40.324805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4446a-39df-4b86-a4f6-b4b0ed174314 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:40.325088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4446a-39df-4b86-a4f6-b4b0ed174314 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:40.325362 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4446a-39df-4b86-a4f6-b4b0ed174314 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:40.325992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4446a-39df-4b86-a4f6-b4b0ed174314 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:40.328029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4446a-39df-4b86-a4f6-b4b0ed174314 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:40.354510 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1003/1985] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:51:40 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:51:40.359009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1a1d5-1df3-4ad6-adff-44d5c39b4987 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:40.369202 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1a1d5-1df3-4ad6-adff-44d5c39b4987 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:40.424459 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d5b94-559e-4ec4-bcf7-a36a6236ca75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:40.425066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d5b94-559e-4ec4-bcf7-a36a6236ca75 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:40.425350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d5b94-559e-4ec4-bcf7-a36a6236ca75 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:40.425629 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d5b94-559e-4ec4-bcf7-a36a6236ca75 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:40.426254 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d5b94-559e-4ec4-bcf7-a36a6236ca75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:40.428238 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d5b94-559e-4ec4-bcf7-a36a6236ca75 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:40.450133 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 983/1986] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:51:40 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:51:40.454958 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4446a-39df-4b86-a4f6-b4b0ed174314 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:40.465177 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4446a-39df-4b86-a4f6-b4b0ed174314 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:40.470366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9dd175-a990-439d-bd08-8272c9f3adfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:40.470960 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9dd175-a990-439d-bd08-8272c9f3adfb admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:40.471239 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9dd175-a990-439d-bd08-8272c9f3adfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:40.471512 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9dd175-a990-439d-bd08-8272c9f3adfb admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:40.472273 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9dd175-a990-439d-bd08-8272c9f3adfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:40.474176 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9dd175-a990-439d-bd08-8272c9f3adfb admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:40.486101 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1004/1987] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:51:40.490073 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d5b94-559e-4ec4-bcf7-a36a6236ca75 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:40.500232 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d5b94-559e-4ec4-bcf7-a36a6236ca75 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:40.504921 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c8e61-a7cb-4877-a95a-7d9049b907a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:40.505525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c8e61-a7cb-4877-a95a-7d9049b907a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:40.505805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c8e61-a7cb-4877-a95a-7d9049b907a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:40.506080 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c8e61-a7cb-4877-a95a-7d9049b907a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:40.506712 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c8e61-a7cb-4877-a95a-7d9049b907a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:40.508348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c8e61-a7cb-4877-a95a-7d9049b907a8 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:40.519532 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 984/1988] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:51:40.523550 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9dd175-a990-439d-bd08-8272c9f3adfb admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:40.533874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9dd175-a990-439d-bd08-8272c9f3adfb admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:40.538416 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f550ab-1128-44d9-8cf0-05fe5e0ede89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:40.539058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f550ab-1128-44d9-8cf0-05fe5e0ede89 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:40.539337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f550ab-1128-44d9-8cf0-05fe5e0ede89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:40.539617 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f550ab-1128-44d9-8cf0-05fe5e0ede89 admin admin] PATH_INFO: `/v3/projects/68d0e30075684bc9a0d40d779c1ad73f/users/4b3fa11eb591443e85f7317b6dc9b304/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:40.554499 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f550ab-1128-44d9-8cf0-05fe5e0ede89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68d0e30075684bc9a0d40d779c1ad73f, user_id=4b3fa11eb591443e85f7317b6dc9b304, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:40.557187 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f550ab-1128-44d9-8cf0-05fe5e0ede89 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:40.572254 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1005/1989] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:51:40 2022] PUT /identity/v3/projects/68d0e30075684bc9a0d40d779c1ad73f/users/4b3fa11eb591443e85f7317b6dc9b304/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:51:40.577903 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe49f3-8e4e-4a28-bf83-fb865eb81b18 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:40.578194 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe49f3-8e4e-4a28-bf83-fb865eb81b18 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:40.578500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe49f3-8e4e-4a28-bf83-fb865eb81b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:40.597918 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-57fe49f3-8e4e-4a28-bf83-fb865eb81b18 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:40.599168 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-57fe49f3-8e4e-4a28-bf83-fb865eb81b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:40.624727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:40.663711 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 985/1990] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:40.673559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f550ab-1128-44d9-8cf0-05fe5e0ede89 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:40.675375 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-72f550ab-1128-44d9-8cf0-05fe5e0ede89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:40.712577 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:40.738676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f550ab-1128-44d9-8cf0-05fe5e0ede89 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:40.742385 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c342e7-efe6-4478-9c13-b279a55bf815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:40.743001 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c342e7-efe6-4478-9c13-b279a55bf815 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:40.743284 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c342e7-efe6-4478-9c13-b279a55bf815 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:40.743559 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c342e7-efe6-4478-9c13-b279a55bf815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:40.753894 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c342e7-efe6-4478-9c13-b279a55bf815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:40.755362 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c342e7-efe6-4478-9c13-b279a55bf815 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:40.765683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1006/1991] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:51:41.360070 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db54b8-0a39-45cd-ade0-0e798664091e None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:41.360625 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db54b8-0a39-45cd-ade0-0e798664091e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:41.360625 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db54b8-0a39-45cd-ade0-0e798664091e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:41.388272 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-99db54b8-0a39-45cd-ade0-0e798664091e None None] MFA Rules not processed for user `4b3fa11eb591443e85f7317b6dc9b304`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:41.389809 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-99db54b8-0a39-45cd-ade0-0e798664091e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:41.457743 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 986/1992] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:41.476527 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c342e7-efe6-4478-9c13-b279a55bf815 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:51:41.478204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-87c342e7-efe6-4478-9c13-b279a55bf815 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:41.540506 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c342e7-efe6-4478-9c13-b279a55bf815 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:51:41.543859 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac13022-59e5-4fb0-afe0-57c2424600c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:51:41.544517 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac13022-59e5-4fb0-afe0-57c2424600c1 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:41.544795 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac13022-59e5-4fb0-afe0-57c2424600c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:41.545064 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac13022-59e5-4fb0-afe0-57c2424600c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:41.554266 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac13022-59e5-4fb0-afe0-57c2424600c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:51:41.555690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac13022-59e5-4fb0-afe0-57c2424600c1 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:51:41.564514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1007/1993] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:51:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:51:45.077865 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8631c-15c2-40a3-90ba-09b5109dcbad None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:51:45.078383 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8631c-15c2-40a3-90ba-09b5109dcbad None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:51:45.078383 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be8631c-15c2-40a3-90ba-09b5109dcbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:51:45.098353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-5be8631c-15c2-40a3-90ba-09b5109dcbad None None] MFA Rules not processed for user `4b3fa11eb591443e85f7317b6dc9b304`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:51:45.099548 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-5be8631c-15c2-40a3-90ba-09b5109dcbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:51:45.106577 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 987/1994] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:51:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:51:51.560455 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=38 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:51:55.093421 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=49 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:02.854897 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac13022-59e5-4fb0-afe0-57c2424600c1 service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:02.856321 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-5ac13022-59e5-4fb0-afe0-57c2424600c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:02.870611 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac13022-59e5-4fb0-afe0-57c2424600c1 service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:02.875875 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0eb7e1-90d0-4938-88e0-0d19f000aa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:02.876567 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0eb7e1-90d0-4938-88e0-0d19f000aa0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:02.876841 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0eb7e1-90d0-4938-88e0-0d19f000aa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:02.877138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0eb7e1-90d0-4938-88e0-0d19f000aa0d admin admin] PATH_INFO: `/v3/users/55e6be4152254f668805883f56e6da53` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:02.878253 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0eb7e1-90d0-4938-88e0-0d19f000aa0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=55e6be4152254f668805883f56e6da53)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:02.880215 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0eb7e1-90d0-4938-88e0-0d19f000aa0d admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:02.931912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7a0eb7e1-90d0-4938-88e0-0d19f000aa0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55e6be4152254f668805883f56e6da53'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:52:02.932662 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7a0eb7e1-90d0-4938-88e0-0d19f000aa0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55e6be4152254f668805883f56e6da53'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:52:02.945161 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7a0eb7e1-90d0-4938-88e0-0d19f000aa0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55e6be4152254f668805883f56e6da53'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:52:02.954492 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7a0eb7e1-90d0-4938-88e0-0d19f000aa0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55e6be4152254f668805883f56e6da53'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:52:02.966624 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7a0eb7e1-90d0-4938-88e0-0d19f000aa0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55e6be4152254f668805883f56e6da53'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:52:02.977452 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1008/1995] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:52:02 2022] DELETE /identity/v3/users/55e6be4152254f668805883f56e6da53 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:52:02.979160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22cfc2a-7d5c-4a0c-adfa-87a713df0828 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:02.979642 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22cfc2a-7d5c-4a0c-adfa-87a713df0828 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:02.979740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22cfc2a-7d5c-4a0c-adfa-87a713df0828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:03.001770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-d22cfc2a-7d5c-4a0c-adfa-87a713df0828 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:52:03.003007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d22cfc2a-7d5c-4a0c-adfa-87a713df0828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:03.070391 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 988/1996] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:52:03.080585 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0eb7e1-90d0-4938-88e0-0d19f000aa0d admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:03.081996 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-7a0eb7e1-90d0-4938-88e0-0d19f000aa0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:03.140741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0eb7e1-90d0-4938-88e0-0d19f000aa0d admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:03.144407 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4e866-5fea-4faa-9c0e-c6b017adabdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:03.145025 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4e866-5fea-4faa-9c0e-c6b017adabdd service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:03.145305 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4e866-5fea-4faa-9c0e-c6b017adabdd service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:03.145579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4e866-5fea-4faa-9c0e-c6b017adabdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:03.154810 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4e866-5fea-4faa-9c0e-c6b017adabdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:03.156638 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4e866-5fea-4faa-9c0e-c6b017adabdd service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:03.166395 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1009/1997] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:52:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:52:03.486279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22cfc2a-7d5c-4a0c-adfa-87a713df0828 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:03.487754 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d22cfc2a-7d5c-4a0c-adfa-87a713df0828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:03.500627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22cfc2a-7d5c-4a0c-adfa-87a713df0828 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:03.505187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84304e9a-d225-417b-ba33-1a66aafb0863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:03.505828 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84304e9a-d225-417b-ba33-1a66aafb0863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:03.506108 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84304e9a-d225-417b-ba33-1a66aafb0863 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:03.506381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84304e9a-d225-417b-ba33-1a66aafb0863 admin admin] PATH_INFO: `/v3/projects/feb686356f3244e9b69153cda8953c99` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:03.507492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84304e9a-d225-417b-ba33-1a66aafb0863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=feb686356f3244e9b69153cda8953c99)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:03.509457 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84304e9a-d225-417b-ba33-1a66aafb0863 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:03.557602 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-84304e9a-d225-417b-ba33-1a66aafb0863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feb686356f3244e9b69153cda8953c99', 'request_id': 'req-84304e9a-d225-417b-ba33-1a66aafb0863'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:52:03.565665 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-84304e9a-d225-417b-ba33-1a66aafb0863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feb686356f3244e9b69153cda8953c99', 'request_id': 'req-84304e9a-d225-417b-ba33-1a66aafb0863'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:52:03.572189 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-84304e9a-d225-417b-ba33-1a66aafb0863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feb686356f3244e9b69153cda8953c99', 'request_id': 'req-84304e9a-d225-417b-ba33-1a66aafb0863'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:52:03.582991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-84304e9a-d225-417b-ba33-1a66aafb0863 admin admin] The token cache is being invalidate because project feb686356f3244e9b69153cda8953c99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:52:03.583280 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-84304e9a-d225-417b-ba33-1a66aafb0863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:52:03.583910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-84304e9a-d225-417b-ba33-1a66aafb0863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:52:03.585432 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 989/1998] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:52:03 2022] DELETE /identity/v3/projects/feb686356f3244e9b69153cda8953c99 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:52:03.590183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c154170-6855-4454-aed6-c73025799fe6 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:03.590476 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c154170-6855-4454-aed6-c73025799fe6 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:03.590783 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c154170-6855-4454-aed6-c73025799fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:03.611778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-9c154170-6855-4454-aed6-c73025799fe6 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:52:03.613154 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-9c154170-6855-4454-aed6-c73025799fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:03.671696 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1010/1999] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:52:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:52:03.680332 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d27cb3-645b-4a2b-ba68-6a231936689c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:03.680617 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d27cb3-645b-4a2b-ba68-6a231936689c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:03.680888 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d27cb3-645b-4a2b-ba68-6a231936689c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:03.701650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-71d27cb3-645b-4a2b-ba68-6a231936689c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:52:03.703536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-71d27cb3-645b-4a2b-ba68-6a231936689c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:03.711636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 990/2000] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:52:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:52:03.716244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c154170-6855-4454-aed6-c73025799fe6 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:03.727376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c154170-6855-4454-aed6-c73025799fe6 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:03.733848 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940a487-0398-4494-b742-ba7c4ae8afc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:03.734594 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940a487-0398-4494-b742-ba7c4ae8afc6 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:03.734878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940a487-0398-4494-b742-ba7c4ae8afc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:03.735150 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940a487-0398-4494-b742-ba7c4ae8afc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:03.735807 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940a487-0398-4494-b742-ba7c4ae8afc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:52:03.736163 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940a487-0398-4494-b742-ba7c4ae8afc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:03.737998 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940a487-0398-4494-b742-ba7c4ae8afc6 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:03.760916 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1011/2001] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:52:03.761216 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d27cb3-645b-4a2b-ba68-6a231936689c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:03.772061 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d27cb3-645b-4a2b-ba68-6a231936689c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:03.781124 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc183de-d4a9-4063-8417-b5580a55a5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:03.781753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc183de-d4a9-4063-8417-b5580a55a5f2 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:03.782047 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc183de-d4a9-4063-8417-b5580a55a5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:03.782311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc183de-d4a9-4063-8417-b5580a55a5f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:03.782948 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc183de-d4a9-4063-8417-b5580a55a5f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:03.784888 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc183de-d4a9-4063-8417-b5580a55a5f2 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:03.816497 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 991/2002] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:52:03 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:52:03.817996 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940a487-0398-4494-b742-ba7c4ae8afc6 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:03.823541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:03.830912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940a487-0398-4494-b742-ba7c4ae8afc6 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:03.897431 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6870011-c271-4fa1-b62a-8398a43c682b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:03.898062 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6870011-c271-4fa1-b62a-8398a43c682b admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:03.898327 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6870011-c271-4fa1-b62a-8398a43c682b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:03.898604 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6870011-c271-4fa1-b62a-8398a43c682b admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:03.899244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6870011-c271-4fa1-b62a-8398a43c682b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:03.901212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6870011-c271-4fa1-b62a-8398a43c682b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:03.923718 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1012/2003] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:52:03 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:52:03.927812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc183de-d4a9-4063-8417-b5580a55a5f2 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:03.938360 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc183de-d4a9-4063-8417-b5580a55a5f2 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:03.943390 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71852559-48a1-4de9-a4b7-8636750c7d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:03.943990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71852559-48a1-4de9-a4b7-8636750c7d65 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:03.944311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71852559-48a1-4de9-a4b7-8636750c7d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:03.944587 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71852559-48a1-4de9-a4b7-8636750c7d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:03.945227 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71852559-48a1-4de9-a4b7-8636750c7d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:03.946832 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71852559-48a1-4de9-a4b7-8636750c7d65 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:03.956970 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 992/2004] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:52:03.960808 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6870011-c271-4fa1-b62a-8398a43c682b admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:03.971166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6870011-c271-4fa1-b62a-8398a43c682b admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:03.975722 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cf262-fe1c-4a73-ab9b-dd2ca6f26f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:03.976350 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cf262-fe1c-4a73-ab9b-dd2ca6f26f8d admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:03.976631 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cf262-fe1c-4a73-ab9b-dd2ca6f26f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:03.976906 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cf262-fe1c-4a73-ab9b-dd2ca6f26f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:03.977545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cf262-fe1c-4a73-ab9b-dd2ca6f26f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:03.979282 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cf262-fe1c-4a73-ab9b-dd2ca6f26f8d admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:03.991852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1013/2005] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:52:03.996042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71852559-48a1-4de9-a4b7-8636750c7d65 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:04.006214 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71852559-48a1-4de9-a4b7-8636750c7d65 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:04.010750 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4374b-a2f7-4665-a284-0d6a58b75727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:04.011387 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4374b-a2f7-4665-a284-0d6a58b75727 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:04.011672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4374b-a2f7-4665-a284-0d6a58b75727 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:04.011944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4374b-a2f7-4665-a284-0d6a58b75727 admin admin] PATH_INFO: `/v3/projects/d870a579f94748d1998b1902b9dc8636/users/b8b8683a89f043b792dc6ba73af2d22b/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:04.031883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4374b-a2f7-4665-a284-0d6a58b75727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d870a579f94748d1998b1902b9dc8636, user_id=b8b8683a89f043b792dc6ba73af2d22b, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:04.036564 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4374b-a2f7-4665-a284-0d6a58b75727 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:04.050207 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 993/2006] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:52:03 2022] PUT /identity/v3/projects/d870a579f94748d1998b1902b9dc8636/users/b8b8683a89f043b792dc6ba73af2d22b/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:52:04.055220 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4256f0e-bdb7-493e-bb21-a3e998c88cc5 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:04.055507 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4256f0e-bdb7-493e-bb21-a3e998c88cc5 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:04.055807 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4256f0e-bdb7-493e-bb21-a3e998c88cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:04.075048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-a4256f0e-bdb7-493e-bb21-a3e998c88cc5 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:52:04.076369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a4256f0e-bdb7-493e-bb21-a3e998c88cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:04.135166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1014/2007] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:52:04.144815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4374b-a2f7-4665-a284-0d6a58b75727 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:04.146442 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e6c4374b-a2f7-4665-a284-0d6a58b75727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:04.169620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:04.209790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4374b-a2f7-4665-a284-0d6a58b75727 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:04.213477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbaed7-daab-4791-917d-f82512eb418b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:04.214121 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbaed7-daab-4791-917d-f82512eb418b service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:04.215536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbaed7-daab-4791-917d-f82512eb418b service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:04.215819 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbaed7-daab-4791-917d-f82512eb418b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:04.228157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbaed7-daab-4791-917d-f82512eb418b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:04.229623 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbaed7-daab-4791-917d-f82512eb418b service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:04.241649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 994/2008] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:52:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:52:04.896585 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d339d-2904-4924-a93d-243760673a63 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:04.896872 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d339d-2904-4924-a93d-243760673a63 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:04.897197 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d339d-2904-4924-a93d-243760673a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:04.928512 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-5b6d339d-2904-4924-a93d-243760673a63 None None] MFA Rules not processed for user `b8b8683a89f043b792dc6ba73af2d22b`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:52:04.930563 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-5b6d339d-2904-4924-a93d-243760673a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:05.013365 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1015/2009] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:52:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:52:05.025066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbaed7-daab-4791-917d-f82512eb418b service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:05.026632 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-32fbaed7-daab-4791-917d-f82512eb418b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:05.093805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbaed7-daab-4791-917d-f82512eb418b service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:05.097491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eaab12-59de-46f2-b1c9-deca92d64d3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:05.098109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eaab12-59de-46f2-b1c9-deca92d64d3a service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:05.098393 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eaab12-59de-46f2-b1c9-deca92d64d3a service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:05.098671 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eaab12-59de-46f2-b1c9-deca92d64d3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:05.107987 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eaab12-59de-46f2-b1c9-deca92d64d3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:05.110036 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eaab12-59de-46f2-b1c9-deca92d64d3a service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:05.119474 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 995/2010] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:52:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:52:06.526732 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae30fe4-04a2-4904-9ac1-b58717a17f44 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:06.527197 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae30fe4-04a2-4904-9ac1-b58717a17f44 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:06.527291 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae30fe4-04a2-4904-9ac1-b58717a17f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:06.547358 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-1ae30fe4-04a2-4904-9ac1-b58717a17f44 None None] MFA Rules not processed for user `b8b8683a89f043b792dc6ba73af2d22b`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:52:06.548532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-1ae30fe4-04a2-4904-9ac1-b58717a17f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:06.555552 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1016/2011] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:52:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:52:15.115124 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=43 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:16.541487 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:26.815647 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9df305-b34e-4e1f-9916-39196228c6fd None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:26.816172 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9df305-b34e-4e1f-9916-39196228c6fd None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:26.816277 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9df305-b34e-4e1f-9916-39196228c6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:26.835909 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-6d9df305-b34e-4e1f-9916-39196228c6fd None None] MFA Rules not processed for user `b8b8683a89f043b792dc6ba73af2d22b`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:52:26.837116 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-6d9df305-b34e-4e1f-9916-39196228c6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:26.843745 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 996/2012] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:52:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:52:36.830740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:43.508563 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f29f13-8643-4417-8251-4b88a35d29a7 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:43.509263 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f29f13-8643-4417-8251-4b88a35d29a7 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:43.509549 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f29f13-8643-4417-8251-4b88a35d29a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:43.532434 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-d4f29f13-8643-4417-8251-4b88a35d29a7 None None] MFA Rules not processed for user `8a91d3b9c4834d729211ec367e235889`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:52:43.534168 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d4f29f13-8643-4417-8251-4b88a35d29a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:43.594676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1017/2013] 10.222.0.36 () {58 vars in 1009 bytes} [Sun Oct 30 00:52:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:52:43.643568 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9df305-b34e-4e1f-9916-39196228c6fd None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:52:43.646570 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-6d9df305-b34e-4e1f-9916-39196228c6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:52:43.729854 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9df305-b34e-4e1f-9916-39196228c6fd None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:52:43.745349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5743dc-ac40-475d-a675-e3fce09c56e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f4c779b8422432fb06cdf996841b9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:52:43.747022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5743dc-ac40-475d-a675-e3fce09c56e1 service swift] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:52:43.747680 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5743dc-ac40-475d-a675-e3fce09c56e1 service swift] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:52:43.748149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5743dc-ac40-475d-a675-e3fce09c56e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:52:43.766246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5743dc-ac40-475d-a675-e3fce09c56e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:52:43.768130 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5743dc-ac40-475d-a675-e3fce09c56e1 service swift] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:52:43.792165 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 997/2014] 10.222.0.36 () {60 vars in 1384 bytes} [Sun Oct 30 00:52:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 00:52:53.588207 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=25 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:52:53.773686 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=32 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:53:07.043600 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f29f13-8643-4417-8251-4b88a35d29a7 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:07.045607 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d4f29f13-8643-4417-8251-4b88a35d29a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:07.056921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f29f13-8643-4417-8251-4b88a35d29a7 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:07.060510 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba86b8-247d-4337-90ee-304d17daa8d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:07.061108 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba86b8-247d-4337-90ee-304d17daa8d4 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:07.061389 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba86b8-247d-4337-90ee-304d17daa8d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:07.061663 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba86b8-247d-4337-90ee-304d17daa8d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:07.063529 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-76ba86b8-247d-4337-90ee-304d17daa8d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:07.135284 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba86b8-247d-4337-90ee-304d17daa8d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:07.136695 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba86b8-247d-4337-90ee-304d17daa8d4 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:07.148204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1018/2015] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:53:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:53:09.072596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c8518-9cb6-4079-b637-c89140e7b4ce None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:09.073149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c8518-9cb6-4079-b637-c89140e7b4ce None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:09.073149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c8518-9cb6-4079-b637-c89140e7b4ce None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:09.074582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 998/2016] 10.222.0.36 () {54 vars in 921 bytes} [Sun Oct 30 00:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:53:09.080822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422606c6-2d26-4398-8fdf-63cc17414002 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:09.081912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422606c6-2d26-4398-8fdf-63cc17414002 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:09.082020 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422606c6-2d26-4398-8fdf-63cc17414002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:09.103715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-422606c6-2d26-4398-8fdf-63cc17414002 None None] MFA Rules not processed for user `adbba7be39a64230bab160ce77ce7586`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:53:09.104991 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-422606c6-2d26-4398-8fdf-63cc17414002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:09.114569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1019/2017] 10.222.0.36 () {58 vars in 1018 bytes} [Sun Oct 30 00:53:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:53:09.126028 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c8518-9cb6-4079-b637-c89140e7b4ce None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:09.127740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-194c8518-9cb6-4079-b637-c89140e7b4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:09.187979 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c8518-9cb6-4079-b637-c89140e7b4ce None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:09.191916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc54c7f-260a-41f1-8b00-ac5bd5934965 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cf9451f2ef7478589b4a4278cb03252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:09.192775 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc54c7f-260a-41f1-8b00-ac5bd5934965 service glance] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:09.193055 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc54c7f-260a-41f1-8b00-ac5bd5934965 service glance] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:09.193327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc54c7f-260a-41f1-8b00-ac5bd5934965 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:09.204131 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc54c7f-260a-41f1-8b00-ac5bd5934965 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:09.205693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc54c7f-260a-41f1-8b00-ac5bd5934965 service glance] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:09.217032 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 999/2018] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:53:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:53:09.285313 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741f24c-74bd-46eb-94cb-ecb7a71750d6 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:09.285596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741f24c-74bd-46eb-94cb-ecb7a71750d6 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:09.285876 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741f24c-74bd-46eb-94cb-ecb7a71750d6 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:09.287241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1020/2019] 10.222.0.36 () {54 vars in 921 bytes} [Sun Oct 30 00:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:53:09.291923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b4521-c73c-4a91-9442-9d24550dd7b2 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:09.292241 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b4521-c73c-4a91-9442-9d24550dd7b2 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:09.292520 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b4521-c73c-4a91-9442-9d24550dd7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:09.312399 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-2d5b4521-c73c-4a91-9442-9d24550dd7b2 None None] MFA Rules not processed for user `fb39b6cc97a1456bb1312f49118c6e10`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:53:09.313535 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-2d5b4521-c73c-4a91-9442-9d24550dd7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:09.321222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1000/2020] 10.222.0.36 () {58 vars in 1018 bytes} [Sun Oct 30 00:53:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:53:09.333169 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741f24c-74bd-46eb-94cb-ecb7a71750d6 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:09.344050 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741f24c-74bd-46eb-94cb-ecb7a71750d6 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:09.348113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7152e87-ed06-49bc-9a72-708fc54d65f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:09.348794 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7152e87-ed06-49bc-9a72-708fc54d65f6 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:09.349072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7152e87-ed06-49bc-9a72-708fc54d65f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:09.349343 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7152e87-ed06-49bc-9a72-708fc54d65f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:09.356524 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7152e87-ed06-49bc-9a72-708fc54d65f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:09.358076 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7152e87-ed06-49bc-9a72-708fc54d65f6 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:09.366757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1021/2021] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:53:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:53:09.457091 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547d170-a6f3-46f8-b306-f115dca40c64 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:09.457928 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547d170-a6f3-46f8-b306-f115dca40c64 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:09.458263 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547d170-a6f3-46f8-b306-f115dca40c64 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:09.463418 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1001/2022] 10.222.0.36 () {54 vars in 921 bytes} [Sun Oct 30 00:53:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:53:09.470658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ee9b5-6f24-41a1-99a5-3d3646fd89ab None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:09.471071 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ee9b5-6f24-41a1-99a5-3d3646fd89ab None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:09.471517 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ee9b5-6f24-41a1-99a5-3d3646fd89ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:09.494886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-a50ee9b5-6f24-41a1-99a5-3d3646fd89ab None None] MFA Rules not processed for user `adbba7be39a64230bab160ce77ce7586`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:53:09.496149 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a50ee9b5-6f24-41a1-99a5-3d3646fd89ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:09.502762 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5d080-c592-4814-b7df-b543711fe003 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:09.503079 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5d080-c592-4814-b7df-b543711fe003 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:09.503900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5d080-c592-4814-b7df-b543711fe003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:09.511252 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1022/2023] 10.222.0.36 () {58 vars in 1018 bytes} [Sun Oct 30 00:53:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:53:09.523471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ee9b5-6f24-41a1-99a5-3d3646fd89ab None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:09.525509 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-fed5d080-c592-4814-b7df-b543711fe003 None None] MFA Rules not processed for user `8a91d3b9c4834d729211ec367e235889`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:53:09.526970 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-fed5d080-c592-4814-b7df-b543711fe003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:09.533293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ee9b5-6f24-41a1-99a5-3d3646fd89ab None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:09.536968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1002/2024] 10.222.0.36 () {58 vars in 1009 bytes} [Sun Oct 30 00:53:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:53:09.538355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0061a3aa-cd00-4c91-86bb-f90437a0426f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:09.538995 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061a3aa-cd00-4c91-86bb-f90437a0426f service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:09.539276 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061a3aa-cd00-4c91-86bb-f90437a0426f service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:09.539548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061a3aa-cd00-4c91-86bb-f90437a0426f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:09.549010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5d080-c592-4814-b7df-b543711fe003 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:09.558322 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0061a3aa-cd00-4c91-86bb-f90437a0426f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:09.559760 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0061a3aa-cd00-4c91-86bb-f90437a0426f service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:09.564811 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5d080-c592-4814-b7df-b543711fe003 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:09.570592 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21c520-604d-4281-a90f-70e701d6d9c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f4c779b8422432fb06cdf996841b9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:09.571211 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21c520-604d-4281-a90f-70e701d6d9c8 service swift] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:09.571503 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21c520-604d-4281-a90f-70e701d6d9c8 service swift] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:09.571611 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1023/2025] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:53:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:53:09.571845 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21c520-604d-4281-a90f-70e701d6d9c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:09.572442 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0061a3aa-cd00-4c91-86bb-f90437a0426f service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:09.580795 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21c520-604d-4281-a90f-70e701d6d9c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:09.582052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0061a3aa-cd00-4c91-86bb-f90437a0426f service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:09.582983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21c520-604d-4281-a90f-70e701d6d9c8 service swift] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:09.585946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47cce0-c534-42b2-93cd-149a4bb7531e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:09.586557 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47cce0-c534-42b2-93cd-149a4bb7531e service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:09.586836 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47cce0-c534-42b2-93cd-149a4bb7531e service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:09.587108 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47cce0-c534-42b2-93cd-149a4bb7531e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:09.588893 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ec47cce0-c534-42b2-93cd-149a4bb7531e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:09.592512 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1003/2026] 10.222.0.36 () {60 vars in 1384 bytes} [Sun Oct 30 00:53:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 30 00:53:09.609771 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec47cce0-c534-42b2-93cd-149a4bb7531e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:09.611223 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec47cce0-c534-42b2-93cd-149a4bb7531e service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:09.627360 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1024/2027] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:53:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:53:16.039086 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21c520-604d-4281-a90f-70e701d6d9c8 service swift] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:16.054287 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21c520-604d-4281-a90f-70e701d6d9c8 service swift] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:16.058832 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee8c9a-96f5-4269-bddc-c18953bb47b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:16.059468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee8c9a-96f5-4269-bddc-c18953bb47b7 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:16.059748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee8c9a-96f5-4269-bddc-c18953bb47b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:16.060049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee8c9a-96f5-4269-bddc-c18953bb47b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:16.062932 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a4ee8c9a-96f5-4269-bddc-c18953bb47b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:16.143789 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee8c9a-96f5-4269-bddc-c18953bb47b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:16.146210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee8c9a-96f5-4269-bddc-c18953bb47b7 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:16.156418 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1004/2028] 10.222.0.36 () {60 vars in 1396 bytes} [Sun Oct 30 00:53:16 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3264 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:53:16.749870 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed9913-491e-42ed-ad46-27c24c5cd68b None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:16.750447 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed9913-491e-42ed-ad46-27c24c5cd68b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:16.750447 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed9913-491e-42ed-ad46-27c24c5cd68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:16.776598 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-01ed9913-491e-42ed-ad46-27c24c5cd68b None None] MFA Rules not processed for user `8a91d3b9c4834d729211ec367e235889`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:53:16.778257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-01ed9913-491e-42ed-ad46-27c24c5cd68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:16.788470 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1025/2029] 10.222.0.36 () {58 vars in 1009 bytes} [Sun Oct 30 00:53:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:53:16.808218 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee8c9a-96f5-4269-bddc-c18953bb47b7 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:16.826255 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee8c9a-96f5-4269-bddc-c18953bb47b7 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:16.833754 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1186a7-d684-4d1c-96bb-d5bf0e47c10d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f4c779b8422432fb06cdf996841b9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:16.834398 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1186a7-d684-4d1c-96bb-d5bf0e47c10d service swift] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:16.834686 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1186a7-d684-4d1c-96bb-d5bf0e47c10d service swift] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:16.834960 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1186a7-d684-4d1c-96bb-d5bf0e47c10d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:16.857079 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1186a7-d684-4d1c-96bb-d5bf0e47c10d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:16.859067 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1186a7-d684-4d1c-96bb-d5bf0e47c10d service swift] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:16.872833 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1005/2030] 10.222.0.36 () {60 vars in 1384 bytes} [Sun Oct 30 00:53:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 00:53:26.768825 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=61 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:53:26.866774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=72 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:53:27.561847 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8956bd8-565d-4116-98cc-0d1e1c86f784 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:27.562216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8956bd8-565d-4116-98cc-0d1e1c86f784 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:27.562560 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8956bd8-565d-4116-98cc-0d1e1c86f784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:27.583916 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-a8956bd8-565d-4116-98cc-0d1e1c86f784 None None] MFA Rules not processed for user `8a91d3b9c4834d729211ec367e235889`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:53:27.585161 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a8956bd8-565d-4116-98cc-0d1e1c86f784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:27.592370 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1026/2031] 10.222.0.36 () {58 vars in 967 bytes} [Sun Oct 30 00:53:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:53:27.605936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1186a7-d684-4d1c-96bb-d5bf0e47c10d service swift] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:27.619790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1186a7-d684-4d1c-96bb-d5bf0e47c10d service swift] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:27.626798 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e6561-1a0a-4396-b6aa-89222fdaca3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f4c779b8422432fb06cdf996841b9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:27.627660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e6561-1a0a-4396-b6aa-89222fdaca3d service swift] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:27.627964 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e6561-1a0a-4396-b6aa-89222fdaca3d service swift] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:27.628384 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e6561-1a0a-4396-b6aa-89222fdaca3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:27.639149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e6561-1a0a-4396-b6aa-89222fdaca3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:27.640915 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1e6561-1a0a-4396-b6aa-89222fdaca3d service swift] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:27.653019 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1006/2032] 10.222.0.36 () {60 vars in 1384 bytes} [Sun Oct 30 00:53:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 00:53:29.048884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8956bd8-565d-4116-98cc-0d1e1c86f784 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:29.050302 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a8956bd8-565d-4116-98cc-0d1e1c86f784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:29.065950 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8956bd8-565d-4116-98cc-0d1e1c86f784 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:29.073916 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5c03c-f437-4571-97c6-0a692890c6dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:29.074852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5c03c-f437-4571-97c6-0a692890c6dd service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:29.075523 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5c03c-f437-4571-97c6-0a692890c6dd service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:29.076063 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5c03c-f437-4571-97c6-0a692890c6dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:29.078392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e4a5c03c-f437-4571-97c6-0a692890c6dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:29.146613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5c03c-f437-4571-97c6-0a692890c6dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:29.148248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5c03c-f437-4571-97c6-0a692890c6dd service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:29.157226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1027/2033] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:53:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:53:29.961018 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e6561-1a0a-4396-b6aa-89222fdaca3d service swift] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:29.971890 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e6561-1a0a-4396-b6aa-89222fdaca3d service swift] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:29.979754 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fbbf8-e69f-4b58-9c85-0c5c5588b9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:29.980668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fbbf8-e69f-4b58-9c85-0c5c5588b9e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:29.981120 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fbbf8-e69f-4b58-9c85-0c5c5588b9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:29.981541 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fbbf8-e69f-4b58-9c85-0c5c5588b9e0 admin admin] PATH_INFO: `/v3/users/b8b8683a89f043b792dc6ba73af2d22b` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:29.983108 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fbbf8-e69f-4b58-9c85-0c5c5588b9e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8b8683a89f043b792dc6ba73af2d22b)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:29.985679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fbbf8-e69f-4b58-9c85-0c5c5588b9e0 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:30.042712 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-333fbbf8-e69f-4b58-9c85-0c5c5588b9e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8b8683a89f043b792dc6ba73af2d22b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:53:30.043297 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-333fbbf8-e69f-4b58-9c85-0c5c5588b9e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8b8683a89f043b792dc6ba73af2d22b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:53:30.056434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-333fbbf8-e69f-4b58-9c85-0c5c5588b9e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8b8683a89f043b792dc6ba73af2d22b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:53:30.063353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-333fbbf8-e69f-4b58-9c85-0c5c5588b9e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8b8683a89f043b792dc6ba73af2d22b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:53:30.064035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-333fbbf8-e69f-4b58-9c85-0c5c5588b9e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8b8683a89f043b792dc6ba73af2d22b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:53:30.084876 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1007/2034] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:53:29 2022] DELETE /identity/v3/users/b8b8683a89f043b792dc6ba73af2d22b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:53:30.280245 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5c03c-f437-4571-97c6-0a692890c6dd service nova] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:30.298337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5c03c-f437-4571-97c6-0a692890c6dd service nova] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:30.454174 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68d714-3c53-4d28-9e2e-56cef992a693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:30.455490 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68d714-3c53-4d28-9e2e-56cef992a693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:30.456072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68d714-3c53-4d28-9e2e-56cef992a693 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:30.456674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68d714-3c53-4d28-9e2e-56cef992a693 admin admin] PATH_INFO: `/v3/projects/d870a579f94748d1998b1902b9dc8636` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:30.462956 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68d714-3c53-4d28-9e2e-56cef992a693 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d870a579f94748d1998b1902b9dc8636)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:30.466030 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68d714-3c53-4d28-9e2e-56cef992a693 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:30.555253 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ad68d714-3c53-4d28-9e2e-56cef992a693 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd870a579f94748d1998b1902b9dc8636', 'request_id': 'req-ad68d714-3c53-4d28-9e2e-56cef992a693'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:53:30.565530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ad68d714-3c53-4d28-9e2e-56cef992a693 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd870a579f94748d1998b1902b9dc8636', 'request_id': 'req-ad68d714-3c53-4d28-9e2e-56cef992a693'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:53:30.577109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ad68d714-3c53-4d28-9e2e-56cef992a693 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd870a579f94748d1998b1902b9dc8636', 'request_id': 'req-ad68d714-3c53-4d28-9e2e-56cef992a693'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:53:30.591271 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ad68d714-3c53-4d28-9e2e-56cef992a693 admin admin] The token cache is being invalidate because project d870a579f94748d1998b1902b9dc8636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:53:30.591593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ad68d714-3c53-4d28-9e2e-56cef992a693 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:53:30.592378 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ad68d714-3c53-4d28-9e2e-56cef992a693 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:53:30.597462 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1028/2035] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:53:30 2022] DELETE /identity/v3/projects/d870a579f94748d1998b1902b9dc8636 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:53:30.600273 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31721501-04d6-450e-9981-62b6863e7f19 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:30.600558 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31721501-04d6-450e-9981-62b6863e7f19 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:30.601346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31721501-04d6-450e-9981-62b6863e7f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:30.636459 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-31721501-04d6-450e-9981-62b6863e7f19 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:53:30.637753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-31721501-04d6-450e-9981-62b6863e7f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:30.707053 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1008/2036] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:53:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:53:30.715492 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669ed25-4395-405b-b1f1-7ed5b536ddd6 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:30.715772 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669ed25-4395-405b-b1f1-7ed5b536ddd6 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:30.716044 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669ed25-4395-405b-b1f1-7ed5b536ddd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:30.737003 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-8669ed25-4395-405b-b1f1-7ed5b536ddd6 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:53:30.738214 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-8669ed25-4395-405b-b1f1-7ed5b536ddd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:30.747227 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1029/2037] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:53:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:53:30.751422 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31721501-04d6-450e-9981-62b6863e7f19 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:30.761102 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31721501-04d6-450e-9981-62b6863e7f19 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:30.765571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a7dd3-c47a-4072-bea6-cd871b3b5b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:30.766160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a7dd3-c47a-4072-bea6-cd871b3b5b7a admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:30.766434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a7dd3-c47a-4072-bea6-cd871b3b5b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:30.766703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a7dd3-c47a-4072-bea6-cd871b3b5b7a admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:30.767335 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a7dd3-c47a-4072-bea6-cd871b3b5b7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:53:30.767632 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a7dd3-c47a-4072-bea6-cd871b3b5b7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:30.769297 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a7dd3-c47a-4072-bea6-cd871b3b5b7a admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:30.780595 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1009/2038] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:53:30.784891 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669ed25-4395-405b-b1f1-7ed5b536ddd6 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:30.794928 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669ed25-4395-405b-b1f1-7ed5b536ddd6 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:30.799544 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3948ba-c5ae-48e2-9308-6888a916b469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:30.800068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3948ba-c5ae-48e2-9308-6888a916b469 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:30.800369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3948ba-c5ae-48e2-9308-6888a916b469 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:30.800639 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3948ba-c5ae-48e2-9308-6888a916b469 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:30.802404 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3948ba-c5ae-48e2-9308-6888a916b469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:30.804345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3948ba-c5ae-48e2-9308-6888a916b469 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:30.819174 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:53:30.833940 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1030/2039] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:53:30 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:53:30.838258 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a7dd3-c47a-4072-bea6-cd871b3b5b7a admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:30.848925 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a7dd3-c47a-4072-bea6-cd871b3b5b7a admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:30.902399 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:53:30.917277 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806b922-774f-41c9-ae99-995a3dd9e223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:30.917941 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806b922-774f-41c9-ae99-995a3dd9e223 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:30.918221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806b922-774f-41c9-ae99-995a3dd9e223 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:30.918495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806b922-774f-41c9-ae99-995a3dd9e223 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:30.919132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3806b922-774f-41c9-ae99-995a3dd9e223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:30.921248 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3806b922-774f-41c9-ae99-995a3dd9e223 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:30.947979 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1010/2040] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:53:30 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:53:30.949428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3948ba-c5ae-48e2-9308-6888a916b469 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:30.961622 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3948ba-c5ae-48e2-9308-6888a916b469 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:30.967486 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810106c-f583-4c3c-b560-7c18d950a30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:30.968300 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810106c-f583-4c3c-b560-7c18d950a30a admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:30.968749 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810106c-f583-4c3c-b560-7c18d950a30a admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:30.969671 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810106c-f583-4c3c-b560-7c18d950a30a admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:30.969913 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810106c-f583-4c3c-b560-7c18d950a30a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:30.971567 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810106c-f583-4c3c-b560-7c18d950a30a admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:30.983487 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1031/2041] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:53:30.987517 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806b922-774f-41c9-ae99-995a3dd9e223 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:30.998415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806b922-774f-41c9-ae99-995a3dd9e223 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:31.003033 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7ee73-776f-46bb-a171-4c5d21967e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:31.003636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7ee73-776f-46bb-a171-4c5d21967e2e admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:31.003913 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7ee73-776f-46bb-a171-4c5d21967e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:31.004222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7ee73-776f-46bb-a171-4c5d21967e2e admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:31.005420 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c7ee73-776f-46bb-a171-4c5d21967e2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:31.007017 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c7ee73-776f-46bb-a171-4c5d21967e2e admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:31.017929 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1011/2042] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:53:31.023502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810106c-f583-4c3c-b560-7c18d950a30a admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:31.033574 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810106c-f583-4c3c-b560-7c18d950a30a admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:31.038542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f13179-1271-4376-9b59-1ab3e3d2212d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:31.039183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f13179-1271-4376-9b59-1ab3e3d2212d admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:31.039462 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f13179-1271-4376-9b59-1ab3e3d2212d admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:31.039736 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f13179-1271-4376-9b59-1ab3e3d2212d admin admin] PATH_INFO: `/v3/projects/12b6e06859f24f798b222c646238b39d/users/d9eee66bc32a4bf09c786f1227258c62/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:31.054380 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f13179-1271-4376-9b59-1ab3e3d2212d admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b6e06859f24f798b222c646238b39d, user_id=d9eee66bc32a4bf09c786f1227258c62, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:31.057009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f13179-1271-4376-9b59-1ab3e3d2212d admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:31.070413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1032/2043] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:53:31 2022] PUT /identity/v3/projects/12b6e06859f24f798b222c646238b39d/users/d9eee66bc32a4bf09c786f1227258c62/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:53:31.080952 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c5692-f8f9-4498-b5bb-fe808543f2e5 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:31.081234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c5692-f8f9-4498-b5bb-fe808543f2e5 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:31.081507 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c5692-f8f9-4498-b5bb-fe808543f2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:31.109486 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-965c5692-f8f9-4498-b5bb-fe808543f2e5 None None] MFA Rules not processed for user `d9eee66bc32a4bf09c786f1227258c62`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:53:31.110684 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-965c5692-f8f9-4498-b5bb-fe808543f2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:31.173066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1012/2044] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:53:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:53:31.184741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f13179-1271-4376-9b59-1ab3e3d2212d admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:31.186483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-75f13179-1271-4376-9b59-1ab3e3d2212d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:31.242714 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f13179-1271-4376-9b59-1ab3e3d2212d admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:31.246345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e50744-01b4-425d-848c-1a015e045f34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:31.246941 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e50744-01b4-425d-848c-1a015e045f34 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:31.247219 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e50744-01b4-425d-848c-1a015e045f34 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:31.247491 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e50744-01b4-425d-848c-1a015e045f34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:31.256578 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e50744-01b4-425d-848c-1a015e045f34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:31.257992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e50744-01b4-425d-848c-1a015e045f34 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:31.266555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1033/2045] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:53:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:53:35.367116 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c5692-f8f9-4498-b5bb-fe808543f2e5 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:35.368728 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-965c5692-f8f9-4498-b5bb-fe808543f2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:35.430115 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c5692-f8f9-4498-b5bb-fe808543f2e5 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:35.434022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ecc62-432b-46da-9a9f-b463ab7c194b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:35.434625 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ecc62-432b-46da-9a9f-b463ab7c194b service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:35.434901 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ecc62-432b-46da-9a9f-b463ab7c194b service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:35.435178 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ecc62-432b-46da-9a9f-b463ab7c194b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:35.436896 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e36ecc62-432b-46da-9a9f-b463ab7c194b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:35.447426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36ecc62-432b-46da-9a9f-b463ab7c194b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:35.448924 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36ecc62-432b-46da-9a9f-b463ab7c194b service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:35.457178 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1013/2046] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:53:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:53:36.127438 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde38e4-9e99-4619-b8f0-ca1e63ff8154 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:36.127946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde38e4-9e99-4619-b8f0-ca1e63ff8154 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:36.128052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde38e4-9e99-4619-b8f0-ca1e63ff8154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:36.146770 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-6fde38e4-9e99-4619-b8f0-ca1e63ff8154 None None] MFA Rules not processed for user `4b3fa11eb591443e85f7317b6dc9b304`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:53:36.147963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-6fde38e4-9e99-4619-b8f0-ca1e63ff8154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:36.203183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1034/2047] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:53:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:53:36.844720 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59a03f-5d36-4c14-8ce9-ac7e5226dee2 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:36.845283 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59a03f-5d36-4c14-8ce9-ac7e5226dee2 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:36.845283 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59a03f-5d36-4c14-8ce9-ac7e5226dee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:36.868053 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-2c59a03f-5d36-4c14-8ce9-ac7e5226dee2 None None] MFA Rules not processed for user `d9eee66bc32a4bf09c786f1227258c62`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:53:36.868724 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-2c59a03f-5d36-4c14-8ce9-ac7e5226dee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:36.884336 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1014/2048] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:53:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:53:36.884807 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde38e4-9e99-4619-b8f0-ca1e63ff8154 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:36.886239 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-6fde38e4-9e99-4619-b8f0-ca1e63ff8154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:36.984664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde38e4-9e99-4619-b8f0-ca1e63ff8154 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:36.991101 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20f711-33d7-442f-b97b-995ffd406550 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:36.991862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20f711-33d7-442f-b97b-995ffd406550 service placement] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:36.992246 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20f711-33d7-442f-b97b-995ffd406550 service placement] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:36.992525 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20f711-33d7-442f-b97b-995ffd406550 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:36.994367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-8f20f711-33d7-442f-b97b-995ffd406550 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:37.012926 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:53:37.016415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20f711-33d7-442f-b97b-995ffd406550 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:37.018640 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20f711-33d7-442f-b97b-995ffd406550 service placement] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:37.030946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1035/2049] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:53:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:53:46.861721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:53:47.027102 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:53:55.191115 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2265f-358f-4c3f-ac8a-13087152f8ad None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:55.191558 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2265f-358f-4c3f-ac8a-13087152f8ad None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:55.191666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2265f-358f-4c3f-ac8a-13087152f8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:55.211884 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-c8c2265f-358f-4c3f-ac8a-13087152f8ad None None] MFA Rules not processed for user `8a91d3b9c4834d729211ec367e235889`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:53:55.213043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c8c2265f-358f-4c3f-ac8a-13087152f8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:55.267214 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1015/2050] 10.222.0.36 () {58 vars in 967 bytes} [Sun Oct 30 00:53:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:53:55.289602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20f711-33d7-442f-b97b-995ffd406550 service placement] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:53:55.291017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-8f20f711-33d7-442f-b97b-995ffd406550 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:53:55.359847 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20f711-33d7-442f-b97b-995ffd406550 service placement] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:53:55.364746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cc2cf-6899-4a1d-b480-e1a8d4146713 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f4c779b8422432fb06cdf996841b9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:53:55.365510 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cc2cf-6899-4a1d-b480-e1a8d4146713 service swift] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:53:55.365791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cc2cf-6899-4a1d-b480-e1a8d4146713 service swift] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:53:55.366061 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cc2cf-6899-4a1d-b480-e1a8d4146713 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:53:55.374054 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2cc2cf-6899-4a1d-b480-e1a8d4146713 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:53:55.375624 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2cc2cf-6899-4a1d-b480-e1a8d4146713 service swift] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:53:55.383650 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1036/2051] 10.222.0.36 () {60 vars in 1384 bytes} [Sun Oct 30 00:53:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 00:54:00.037811 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2265f-358f-4c3f-ac8a-13087152f8ad None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:54:00.039444 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c8c2265f-358f-4c3f-ac8a-13087152f8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:54:00.138315 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2265f-358f-4c3f-ac8a-13087152f8ad None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:54:00.145005 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4988fb-c5ba-43ff-9657-b9258849bd8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:54:00.145915 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4988fb-c5ba-43ff-9657-b9258849bd8b service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:54:00.146294 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4988fb-c5ba-43ff-9657-b9258849bd8b service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:54:00.146559 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4988fb-c5ba-43ff-9657-b9258849bd8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:54:00.149921 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-cc4988fb-c5ba-43ff-9657-b9258849bd8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:54:00.166208 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4988fb-c5ba-43ff-9657-b9258849bd8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:54:00.168169 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4988fb-c5ba-43ff-9657-b9258849bd8b service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:54:00.184811 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1016/2052] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:54:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:54:02.158549 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac965eeb-5910-4d60-a3c3-818a4fbf3248 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:54:02.159056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac965eeb-5910-4d60-a3c3-818a4fbf3248 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:54:02.159157 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac965eeb-5910-4d60-a3c3-818a4fbf3248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:54:02.179232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-ac965eeb-5910-4d60-a3c3-818a4fbf3248 None None] MFA Rules not processed for user `8a91d3b9c4834d729211ec367e235889`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:54:02.180527 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ac965eeb-5910-4d60-a3c3-818a4fbf3248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:54:02.187932 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1037/2053] 10.222.0.36 () {58 vars in 1009 bytes} [Sun Oct 30 00:54:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:54:02.200713 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4988fb-c5ba-43ff-9657-b9258849bd8b service nova] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:54:02.215220 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4988fb-c5ba-43ff-9657-b9258849bd8b service nova] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:54:02.220157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecd4cd-eab2-40a8-979e-c69ff80e93c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f4c779b8422432fb06cdf996841b9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:54:02.220756 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecd4cd-eab2-40a8-979e-c69ff80e93c6 service swift] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:54:02.221029 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecd4cd-eab2-40a8-979e-c69ff80e93c6 service swift] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:54:02.221299 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecd4cd-eab2-40a8-979e-c69ff80e93c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:54:02.229483 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ecd4cd-eab2-40a8-979e-c69ff80e93c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:54:02.231430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ecd4cd-eab2-40a8-979e-c69ff80e93c6 service swift] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:54:02.239607 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1017/2054] 10.222.0.36 () {60 vars in 1384 bytes} [Sun Oct 30 00:54:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 00:54:02.742995 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe14ca-7d78-46eb-b1bd-7102bdae00bf None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:54:02.743348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe14ca-7d78-46eb-b1bd-7102bdae00bf None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:54:02.743630 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe14ca-7d78-46eb-b1bd-7102bdae00bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:54:02.763502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-20fe14ca-7d78-46eb-b1bd-7102bdae00bf None None] MFA Rules not processed for user `8a91d3b9c4834d729211ec367e235889`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:54:02.764719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-20fe14ca-7d78-46eb-b1bd-7102bdae00bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:54:02.771779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1038/2055] 10.222.0.36 () {58 vars in 967 bytes} [Sun Oct 30 00:54:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:54:02.784315 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecd4cd-eab2-40a8-979e-c69ff80e93c6 service swift] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:54:02.795053 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecd4cd-eab2-40a8-979e-c69ff80e93c6 service swift] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:54:02.800017 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a286a8-c955-47c8-be64-352b29ec9b10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f4c779b8422432fb06cdf996841b9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:54:02.800683 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a286a8-c955-47c8-be64-352b29ec9b10 service swift] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:54:02.800962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a286a8-c955-47c8-be64-352b29ec9b10 service swift] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:54:02.801235 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a286a8-c955-47c8-be64-352b29ec9b10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:54:02.809894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a286a8-c955-47c8-be64-352b29ec9b10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:54:02.811474 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a286a8-c955-47c8-be64-352b29ec9b10 service swift] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:54:02.819983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1018/2056] 10.222.0.36 () {60 vars in 1384 bytes} [Sun Oct 30 00:54:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 00:54:03.042245 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83f282-2f5c-4dbc-8b63-5a8898fa867f None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:54:03.042529 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83f282-2f5c-4dbc-8b63-5a8898fa867f None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:54:03.042802 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83f282-2f5c-4dbc-8b63-5a8898fa867f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:54:03.062715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-6a83f282-2f5c-4dbc-8b63-5a8898fa867f None None] MFA Rules not processed for user `8a91d3b9c4834d729211ec367e235889`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:54:03.063919 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-6a83f282-2f5c-4dbc-8b63-5a8898fa867f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:54:03.071368 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1039/2057] 10.222.0.36 () {58 vars in 1009 bytes} [Sun Oct 30 00:54:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:54:03.083557 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a286a8-c955-47c8-be64-352b29ec9b10 service swift] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:54:03.094225 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a286a8-c955-47c8-be64-352b29ec9b10 service swift] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:54:03.099388 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638f841-d4f6-44d7-92ac-6f1a3226752b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f4c779b8422432fb06cdf996841b9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:54:03.100008 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638f841-d4f6-44d7-92ac-6f1a3226752b service swift] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:54:03.100310 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638f841-d4f6-44d7-92ac-6f1a3226752b service swift] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:54:03.101195 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638f841-d4f6-44d7-92ac-6f1a3226752b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:54:03.109544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8638f841-d4f6-44d7-92ac-6f1a3226752b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:54:03.111118 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8638f841-d4f6-44d7-92ac-6f1a3226752b service swift] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:54:03.119669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1019/2058] 10.222.0.36 () {60 vars in 1384 bytes} [Sun Oct 30 00:54:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 00:54:03.443413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a3441-a9a0-4e2c-9943-24d12efc731b None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:54:03.443905 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a3441-a9a0-4e2c-9943-24d12efc731b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:54:03.444004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a3441-a9a0-4e2c-9943-24d12efc731b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:54:03.463737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-be6a3441-a9a0-4e2c-9943-24d12efc731b None None] MFA Rules not processed for user `8a91d3b9c4834d729211ec367e235889`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:54:03.464970 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-be6a3441-a9a0-4e2c-9943-24d12efc731b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:54:03.472032 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1040/2059] 10.222.0.36 () {58 vars in 967 bytes} [Sun Oct 30 00:54:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:54:03.485162 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638f841-d4f6-44d7-92ac-6f1a3226752b service swift] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:54:03.495843 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638f841-d4f6-44d7-92ac-6f1a3226752b service swift] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:54:03.500843 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce494b6d-7986-4b68-8eba-189b2a1a6027 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f4c779b8422432fb06cdf996841b9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:54:03.501435 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce494b6d-7986-4b68-8eba-189b2a1a6027 service swift] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:54:03.501713 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce494b6d-7986-4b68-8eba-189b2a1a6027 service swift] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:54:03.501984 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce494b6d-7986-4b68-8eba-189b2a1a6027 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:54:03.510051 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce494b6d-7986-4b68-8eba-189b2a1a6027 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:54:03.511630 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce494b6d-7986-4b68-8eba-189b2a1a6027 service swift] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:54:03.519504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1020/2060] 10.222.0.36 () {60 vars in 1384 bytes} [Sun Oct 30 00:54:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 00:54:05.662751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a3441-a9a0-4e2c-9943-24d12efc731b None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:54:05.728877 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a3441-a9a0-4e2c-9943-24d12efc731b None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:54:05.733265 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889356d8-ef1d-4d84-8725-4dc0d7b13ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:54:05.733892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889356d8-ef1d-4d84-8725-4dc0d7b13ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:54:05.734169 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889356d8-ef1d-4d84-8725-4dc0d7b13ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:54:05.734439 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889356d8-ef1d-4d84-8725-4dc0d7b13ea1 admin admin] PATH_INFO: `/v3/users/d9eee66bc32a4bf09c786f1227258c62` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:54:05.735540 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889356d8-ef1d-4d84-8725-4dc0d7b13ea1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9eee66bc32a4bf09c786f1227258c62)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:54:05.737482 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889356d8-ef1d-4d84-8725-4dc0d7b13ea1 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:54:05.789642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-889356d8-ef1d-4d84-8725-4dc0d7b13ea1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9eee66bc32a4bf09c786f1227258c62'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:05.790296 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-889356d8-ef1d-4d84-8725-4dc0d7b13ea1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9eee66bc32a4bf09c786f1227258c62'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:05.802553 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-889356d8-ef1d-4d84-8725-4dc0d7b13ea1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9eee66bc32a4bf09c786f1227258c62'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:05.803719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:54:05.811045 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-889356d8-ef1d-4d84-8725-4dc0d7b13ea1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9eee66bc32a4bf09c786f1227258c62'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:05.818451 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-889356d8-ef1d-4d84-8725-4dc0d7b13ea1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9eee66bc32a4bf09c786f1227258c62'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:05.825013 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1041/2061] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:54:05 2022] DELETE /identity/v3/users/d9eee66bc32a4bf09c786f1227258c62 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:54:05.829545 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d711d0d-1aa3-43bf-b773-5bd8238d6ed0 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:54:05.830049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d711d0d-1aa3-43bf-b773-5bd8238d6ed0 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:54:05.830147 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d711d0d-1aa3-43bf-b773-5bd8238d6ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:54:05.849830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-0d711d0d-1aa3-43bf-b773-5bd8238d6ed0 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:54:05.851109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-0d711d0d-1aa3-43bf-b773-5bd8238d6ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:54:05.886560 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:54:05.914729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1021/2062] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:54:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:54:05.924328 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889356d8-ef1d-4d84-8725-4dc0d7b13ea1 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:54:05.925709 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-889356d8-ef1d-4d84-8725-4dc0d7b13ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:54:05.983796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889356d8-ef1d-4d84-8725-4dc0d7b13ea1 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:54:05.987388 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc37fcc-bccd-4e48-84c9-85d2b46e43bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:54:05.987988 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc37fcc-bccd-4e48-84c9-85d2b46e43bb service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:54:05.988303 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc37fcc-bccd-4e48-84c9-85d2b46e43bb service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:54:05.988577 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc37fcc-bccd-4e48-84c9-85d2b46e43bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:54:05.998063 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc37fcc-bccd-4e48-84c9-85d2b46e43bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:54:05.999479 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc37fcc-bccd-4e48-84c9-85d2b46e43bb service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:54:06.009019 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1042/2063] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:54:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:54:06.180175 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d711d0d-1aa3-43bf-b773-5bd8238d6ed0 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:54:06.181805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-0d711d0d-1aa3-43bf-b773-5bd8238d6ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:54:06.194037 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d711d0d-1aa3-43bf-b773-5bd8238d6ed0 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:54:06.200006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f634f4-5214-4b5b-acd3-4fe328c910d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:54:06.200786 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f634f4-5214-4b5b-acd3-4fe328c910d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:54:06.201133 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f634f4-5214-4b5b-acd3-4fe328c910d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:54:06.201416 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f634f4-5214-4b5b-acd3-4fe328c910d9 admin admin] PATH_INFO: `/v3/projects/12b6e06859f24f798b222c646238b39d` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:54:06.202627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f634f4-5214-4b5b-acd3-4fe328c910d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b6e06859f24f798b222c646238b39d)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:54:06.204823 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f634f4-5214-4b5b-acd3-4fe328c910d9 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:54:06.252828 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-56f634f4-5214-4b5b-acd3-4fe328c910d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b6e06859f24f798b222c646238b39d', 'request_id': 'req-56f634f4-5214-4b5b-acd3-4fe328c910d9'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:06.260129 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-56f634f4-5214-4b5b-acd3-4fe328c910d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b6e06859f24f798b222c646238b39d', 'request_id': 'req-56f634f4-5214-4b5b-acd3-4fe328c910d9'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:06.266952 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-56f634f4-5214-4b5b-acd3-4fe328c910d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b6e06859f24f798b222c646238b39d', 'request_id': 'req-56f634f4-5214-4b5b-acd3-4fe328c910d9'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:06.277689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-56f634f4-5214-4b5b-acd3-4fe328c910d9 admin admin] The token cache is being invalidate because project 12b6e06859f24f798b222c646238b39d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:54:06.278024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-56f634f4-5214-4b5b-acd3-4fe328c910d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:06.278662 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-56f634f4-5214-4b5b-acd3-4fe328c910d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:06.280170 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1022/2064] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:54:06 2022] DELETE /identity/v3/projects/12b6e06859f24f798b222c646238b39d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:54:16.004521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=32,keystone:INSERT=1,keystone:DELETE=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:54:16.274539 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=37,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:54:38.435641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc37fcc-bccd-4e48-84c9-85d2b46e43bb service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:54:38.437155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-3dc37fcc-bccd-4e48-84c9-85d2b46e43bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:54:38.511581 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc37fcc-bccd-4e48-84c9-85d2b46e43bb service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:54:38.575735 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d23b1c-e25b-4ef3-b37b-6de9bd50a113 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:54:38.576378 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d23b1c-e25b-4ef3-b37b-6de9bd50a113 service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:54:38.576655 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d23b1c-e25b-4ef3-b37b-6de9bd50a113 service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:54:38.576927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d23b1c-e25b-4ef3-b37b-6de9bd50a113 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:54:38.578638 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e8d23b1c-e25b-4ef3-b37b-6de9bd50a113 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:54:38.637596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d23b1c-e25b-4ef3-b37b-6de9bd50a113 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:54:38.639192 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d23b1c-e25b-4ef3-b37b-6de9bd50a113 service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:54:38.648195 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1043/2065] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:54:38 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:54:48.643973 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=84 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:54:52.237916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f634f4-5214-4b5b-acd3-4fe328c910d9 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:54:52.240135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-56f634f4-5214-4b5b-acd3-4fe328c910d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:54:52.314450 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f634f4-5214-4b5b-acd3-4fe328c910d9 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:54:52.319314 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172aac2-ca65-4292-beb5-9332668e77ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:54:52.320022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172aac2-ca65-4292-beb5-9332668e77ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:54:52.320719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172aac2-ca65-4292-beb5-9332668e77ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:54:52.321225 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172aac2-ca65-4292-beb5-9332668e77ed admin admin] PATH_INFO: `/v3/users/4b3fa11eb591443e85f7317b6dc9b304` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:54:52.322458 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172aac2-ca65-4292-beb5-9332668e77ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b3fa11eb591443e85f7317b6dc9b304)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:54:52.324380 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172aac2-ca65-4292-beb5-9332668e77ed admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:54:52.376006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-c172aac2-ca65-4292-beb5-9332668e77ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b3fa11eb591443e85f7317b6dc9b304'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:52.376715 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-c172aac2-ca65-4292-beb5-9332668e77ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b3fa11eb591443e85f7317b6dc9b304'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:52.391851 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-c172aac2-ca65-4292-beb5-9332668e77ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b3fa11eb591443e85f7317b6dc9b304'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:52.399007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-c172aac2-ca65-4292-beb5-9332668e77ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b3fa11eb591443e85f7317b6dc9b304'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:52.399696 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-c172aac2-ca65-4292-beb5-9332668e77ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b3fa11eb591443e85f7317b6dc9b304'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:52.413149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1023/2066] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:54:52 2022] DELETE /identity/v3/users/4b3fa11eb591443e85f7317b6dc9b304 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:54:52.582240 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d23b1c-e25b-4ef3-b37b-6de9bd50a113 service nova] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:54:52.595333 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d23b1c-e25b-4ef3-b37b-6de9bd50a113 service nova] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:54:52.653139 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d62fd-133a-4633-8dd5-169a4e85a8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:54:52.654126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d62fd-133a-4633-8dd5-169a4e85a8e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:54:52.654403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d62fd-133a-4633-8dd5-169a4e85a8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:54:52.654670 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d62fd-133a-4633-8dd5-169a4e85a8e4 admin admin] PATH_INFO: `/v3/projects/68d0e30075684bc9a0d40d779c1ad73f` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:54:52.655780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d62fd-133a-4633-8dd5-169a4e85a8e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68d0e30075684bc9a0d40d779c1ad73f)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:54:52.657746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d62fd-133a-4633-8dd5-169a4e85a8e4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:54:52.704476 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ff1d62fd-133a-4633-8dd5-169a4e85a8e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68d0e30075684bc9a0d40d779c1ad73f', 'request_id': 'req-ff1d62fd-133a-4633-8dd5-169a4e85a8e4'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:52.711470 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ff1d62fd-133a-4633-8dd5-169a4e85a8e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68d0e30075684bc9a0d40d779c1ad73f', 'request_id': 'req-ff1d62fd-133a-4633-8dd5-169a4e85a8e4'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:52.717068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ff1d62fd-133a-4633-8dd5-169a4e85a8e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68d0e30075684bc9a0d40d779c1ad73f', 'request_id': 'req-ff1d62fd-133a-4633-8dd5-169a4e85a8e4'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:52.728304 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ff1d62fd-133a-4633-8dd5-169a4e85a8e4 admin admin] The token cache is being invalidate because project 68d0e30075684bc9a0d40d779c1ad73f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:54:52.728592 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ff1d62fd-133a-4633-8dd5-169a4e85a8e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:52.729232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ff1d62fd-133a-4633-8dd5-169a4e85a8e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:54:52.730737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1044/2067] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:54:52 2022] DELETE /identity/v3/projects/68d0e30075684bc9a0d40d779c1ad73f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:55:02.409571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:55:02.725428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:55:04.163923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c690d5d-8893-418b-9dd0-237ebc22c4ac None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:55:04.164464 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c690d5d-8893-418b-9dd0-237ebc22c4ac None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:55:04.164572 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c690d5d-8893-418b-9dd0-237ebc22c4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:55:04.183275 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-5c690d5d-8893-418b-9dd0-237ebc22c4ac None None] MFA Rules not processed for user `350f5e4238f448949b2c372aabbfc835`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:55:04.184504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-5c690d5d-8893-418b-9dd0-237ebc22c4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:55:04.237928 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1024/2068] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:55:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:55:14.231193 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=25 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:55:22.469091 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d62fd-133a-4633-8dd5-169a4e85a8e4 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:55:22.471133 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ff1d62fd-133a-4633-8dd5-169a4e85a8e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:55:22.584763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d62fd-133a-4633-8dd5-169a4e85a8e4 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:55:22.590901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f41bb5-bd1b-4772-8527-f93e4909e1a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:55:22.592049 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f41bb5-bd1b-4772-8527-f93e4909e1a7 service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:55:22.592398 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f41bb5-bd1b-4772-8527-f93e4909e1a7 service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:55:22.592642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f41bb5-bd1b-4772-8527-f93e4909e1a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:55:22.594556 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-35f41bb5-bd1b-4772-8527-f93e4909e1a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:55:22.607804 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f41bb5-bd1b-4772-8527-f93e4909e1a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:55:22.609488 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f41bb5-bd1b-4772-8527-f93e4909e1a7 service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:55:22.630892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1045/2069] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:55:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:55:26.174315 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a42a49-27c5-475b-aaa7-6ab766d4d680 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:55:26.174855 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a42a49-27c5-475b-aaa7-6ab766d4d680 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:55:26.174855 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a42a49-27c5-475b-aaa7-6ab766d4d680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:55:26.193858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-82a42a49-27c5-475b-aaa7-6ab766d4d680 None None] MFA Rules not processed for user `df1ee15eb3bc468188a9d8c1af46529c`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:55:26.195070 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-82a42a49-27c5-475b-aaa7-6ab766d4d680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:55:26.252830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1025/2070] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:55:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:55:26.650085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f41bb5-bd1b-4772-8527-f93e4909e1a7 service nova] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:55:26.651822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-35f41bb5-bd1b-4772-8527-f93e4909e1a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:55:26.711365 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f41bb5-bd1b-4772-8527-f93e4909e1a7 service nova] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:55:26.715201 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611a77e-003a-4f7c-989d-8db37d6ab5da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:55:26.715843 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611a77e-003a-4f7c-989d-8db37d6ab5da service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:55:26.716157 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611a77e-003a-4f7c-989d-8db37d6ab5da service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:55:26.716464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611a77e-003a-4f7c-989d-8db37d6ab5da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:55:26.718363 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-0611a77e-003a-4f7c-989d-8db37d6ab5da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:55:26.728345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611a77e-003a-4f7c-989d-8db37d6ab5da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:55:26.729779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611a77e-003a-4f7c-989d-8db37d6ab5da service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:55:26.739264 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1046/2071] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:55:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:55:35.738682 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a42a49-27c5-475b-aaa7-6ab766d4d680 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:55:35.752809 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a42a49-27c5-475b-aaa7-6ab766d4d680 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:55:35.758420 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef7230-faeb-4760-b17a-afeda35869ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:55:35.759034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef7230-faeb-4760-b17a-afeda35869ba service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:55:35.759331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef7230-faeb-4760-b17a-afeda35869ba service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:55:35.759591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef7230-faeb-4760-b17a-afeda35869ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:55:35.761415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-1cef7230-faeb-4760-b17a-afeda35869ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:55:35.774546 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef7230-faeb-4760-b17a-afeda35869ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:55:35.776178 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef7230-faeb-4760-b17a-afeda35869ba service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:55:35.788110 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1026/2072] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:55:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:55:36.734880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=60 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:55:45.783417 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=31 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:19.401428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611a77e-003a-4f7c-989d-8db37d6ab5da service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:19.402907 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-0611a77e-003a-4f7c-989d-8db37d6ab5da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:19.442428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611a77e-003a-4f7c-989d-8db37d6ab5da service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:19.446041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab3c62-7818-41dd-8b67-0ac61a8836cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:19.446637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab3c62-7818-41dd-8b67-0ac61a8836cf service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:19.446915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab3c62-7818-41dd-8b67-0ac61a8836cf service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:19.447189 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab3c62-7818-41dd-8b67-0ac61a8836cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:19.448903 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c0ab3c62-7818-41dd-8b67-0ac61a8836cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:19.552539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab3c62-7818-41dd-8b67-0ac61a8836cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:19.553884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab3c62-7818-41dd-8b67-0ac61a8836cf service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:19.572757 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1047/2073] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:56:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:56:23.073872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef7230-faeb-4760-b17a-afeda35869ba service nova] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:23.075511 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-1cef7230-faeb-4760-b17a-afeda35869ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:23.128509 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef7230-faeb-4760-b17a-afeda35869ba service nova] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:23.133488 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4286ad-65e4-4428-aecf-fa6e7240f3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:23.134243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4286ad-65e4-4428-aecf-fa6e7240f3c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:23.134688 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4286ad-65e4-4428-aecf-fa6e7240f3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:23.134967 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4286ad-65e4-4428-aecf-fa6e7240f3c2 admin admin] PATH_INFO: `/v3/users/350f5e4238f448949b2c372aabbfc835` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:23.136152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4286ad-65e4-4428-aecf-fa6e7240f3c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=350f5e4238f448949b2c372aabbfc835)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:23.138132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4286ad-65e4-4428-aecf-fa6e7240f3c2 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:23.188401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-aa4286ad-65e4-4428-aecf-fa6e7240f3c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '350f5e4238f448949b2c372aabbfc835'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:23.189005 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-aa4286ad-65e4-4428-aecf-fa6e7240f3c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '350f5e4238f448949b2c372aabbfc835'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:23.200957 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-aa4286ad-65e4-4428-aecf-fa6e7240f3c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '350f5e4238f448949b2c372aabbfc835'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:23.208160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-aa4286ad-65e4-4428-aecf-fa6e7240f3c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '350f5e4238f448949b2c372aabbfc835'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:23.208827 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-aa4286ad-65e4-4428-aecf-fa6e7240f3c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '350f5e4238f448949b2c372aabbfc835'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:23.222331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1027/2074] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:56:23 2022] DELETE /identity/v3/users/350f5e4238f448949b2c372aabbfc835 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:56:23.401950 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab3c62-7818-41dd-8b67-0ac61a8836cf service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:23.413351 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab3c62-7818-41dd-8b67-0ac61a8836cf service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:23.471005 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9eb97-7a68-4ac0-bec8-492c654cb8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:23.472500 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9eb97-7a68-4ac0-bec8-492c654cb8c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:23.472784 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9eb97-7a68-4ac0-bec8-492c654cb8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:23.473062 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9eb97-7a68-4ac0-bec8-492c654cb8c0 admin admin] PATH_INFO: `/v3/projects/999d27163d544dff836ab7d90355f0d6` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:23.474182 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9eb97-7a68-4ac0-bec8-492c654cb8c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=999d27163d544dff836ab7d90355f0d6)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:23.476156 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9eb97-7a68-4ac0-bec8-492c654cb8c0 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:23.498785 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:23.524672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-2df9eb97-7a68-4ac0-bec8-492c654cb8c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '999d27163d544dff836ab7d90355f0d6', 'request_id': 'req-2df9eb97-7a68-4ac0-bec8-492c654cb8c0'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:23.531874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-2df9eb97-7a68-4ac0-bec8-492c654cb8c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '999d27163d544dff836ab7d90355f0d6', 'request_id': 'req-2df9eb97-7a68-4ac0-bec8-492c654cb8c0'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:23.538210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-2df9eb97-7a68-4ac0-bec8-492c654cb8c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '999d27163d544dff836ab7d90355f0d6', 'request_id': 'req-2df9eb97-7a68-4ac0-bec8-492c654cb8c0'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:23.549096 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-2df9eb97-7a68-4ac0-bec8-492c654cb8c0 admin admin] The token cache is being invalidate because project 999d27163d544dff836ab7d90355f0d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:56:23.549445 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-2df9eb97-7a68-4ac0-bec8-492c654cb8c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:23.550072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-2df9eb97-7a68-4ac0-bec8-492c654cb8c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:23.551624 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1048/2075] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:56:23 2022] DELETE /identity/v3/projects/999d27163d544dff836ab7d90355f0d6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:23.556687 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449288f5-ac5c-47ad-b12d-480668ac9f64 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:23.557005 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449288f5-ac5c-47ad-b12d-480668ac9f64 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:23.557325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449288f5-ac5c-47ad-b12d-480668ac9f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:23.616714 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-449288f5-ac5c-47ad-b12d-480668ac9f64 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:23.617853 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-449288f5-ac5c-47ad-b12d-480668ac9f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:23.676298 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1028/2076] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:23.684915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e52f1f-64b2-46e7-bc10-1be68e25b91a None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:23.685200 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e52f1f-64b2-46e7-bc10-1be68e25b91a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:23.685475 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e52f1f-64b2-46e7-bc10-1be68e25b91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:23.704947 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-e9e52f1f-64b2-46e7-bc10-1be68e25b91a None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:23.707111 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e9e52f1f-64b2-46e7-bc10-1be68e25b91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:23.714874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1049/2077] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:23.719211 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449288f5-ac5c-47ad-b12d-480668ac9f64 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:23.731716 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449288f5-ac5c-47ad-b12d-480668ac9f64 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:23.739461 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba87733-2694-4a52-8e68-838b17c7fade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:23.740623 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba87733-2694-4a52-8e68-838b17c7fade admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:23.741134 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba87733-2694-4a52-8e68-838b17c7fade admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:23.741599 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba87733-2694-4a52-8e68-838b17c7fade admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:23.742437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba87733-2694-4a52-8e68-838b17c7fade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:56:23.742944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba87733-2694-4a52-8e68-838b17c7fade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:23.744650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba87733-2694-4a52-8e68-838b17c7fade admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:23.747084 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:23.758318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1029/2078] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:56:23.762444 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e52f1f-64b2-46e7-bc10-1be68e25b91a None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:23.772291 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e52f1f-64b2-46e7-bc10-1be68e25b91a None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:23.776908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ed7af-fb13-4f54-8bcc-bae905296837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:23.777497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ed7af-fb13-4f54-8bcc-bae905296837 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:23.777772 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ed7af-fb13-4f54-8bcc-bae905296837 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:23.778067 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ed7af-fb13-4f54-8bcc-bae905296837 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:23.778685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ed7af-fb13-4f54-8bcc-bae905296837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:23.780612 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ed7af-fb13-4f54-8bcc-bae905296837 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:23.807664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1050/2079] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:56:23 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:23.811825 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba87733-2694-4a52-8e68-838b17c7fade admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:23.822292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba87733-2694-4a52-8e68-838b17c7fade admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:23.878649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b990e1e1-66c2-4cc7-966e-76ed3278e37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:23.879255 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990e1e1-66c2-4cc7-966e-76ed3278e37a admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:23.879536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990e1e1-66c2-4cc7-966e-76ed3278e37a admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:23.879811 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990e1e1-66c2-4cc7-966e-76ed3278e37a admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:23.880489 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b990e1e1-66c2-4cc7-966e-76ed3278e37a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:23.882421 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b990e1e1-66c2-4cc7-966e-76ed3278e37a admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:23.903482 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1030/2080] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:56:23 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:23.907490 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ed7af-fb13-4f54-8bcc-bae905296837 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:23.917714 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ed7af-fb13-4f54-8bcc-bae905296837 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:23.922349 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71077069-e978-47af-b5e0-852276bae6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:23.922946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71077069-e978-47af-b5e0-852276bae6e4 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:23.923225 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71077069-e978-47af-b5e0-852276bae6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:23.923499 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71077069-e978-47af-b5e0-852276bae6e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:23.924216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71077069-e978-47af-b5e0-852276bae6e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:23.926049 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71077069-e978-47af-b5e0-852276bae6e4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:23.935992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1051/2081] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:56:23.940368 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b990e1e1-66c2-4cc7-966e-76ed3278e37a admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:23.950277 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b990e1e1-66c2-4cc7-966e-76ed3278e37a admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:23.954931 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f89939-1bfa-4c08-93e2-f1c28c963f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:23.956438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f89939-1bfa-4c08-93e2-f1c28c963f1e admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:23.956737 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f89939-1bfa-4c08-93e2-f1c28c963f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:23.957013 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f89939-1bfa-4c08-93e2-f1c28c963f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:23.957653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f89939-1bfa-4c08-93e2-f1c28c963f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:23.959255 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f89939-1bfa-4c08-93e2-f1c28c963f1e admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:23.969821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1031/2082] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:23.973983 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71077069-e978-47af-b5e0-852276bae6e4 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:23.983749 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71077069-e978-47af-b5e0-852276bae6e4 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:23.988629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476e7a2-0069-4527-8b5f-05ed17e882e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:23.989266 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476e7a2-0069-4527-8b5f-05ed17e882e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:23.989546 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476e7a2-0069-4527-8b5f-05ed17e882e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:23.989823 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476e7a2-0069-4527-8b5f-05ed17e882e4 admin admin] PATH_INFO: `/v3/projects/c9d7c91873be42cfbad24838de99e145/users/884aac2daddd438b96d16553e8805fd9/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:24.007250 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476e7a2-0069-4527-8b5f-05ed17e882e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9d7c91873be42cfbad24838de99e145, user_id=884aac2daddd438b96d16553e8805fd9, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:24.010028 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476e7a2-0069-4527-8b5f-05ed17e882e4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:24.023401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1052/2083] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:23 2022] PUT /identity/v3/projects/c9d7c91873be42cfbad24838de99e145/users/884aac2daddd438b96d16553e8805fd9/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:24.033608 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c44a4-2df8-45b6-8095-2e9d1ad6e598 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:24.033904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c44a4-2df8-45b6-8095-2e9d1ad6e598 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:24.034466 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c44a4-2df8-45b6-8095-2e9d1ad6e598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:24.063463 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-456c44a4-2df8-45b6-8095-2e9d1ad6e598 None None] MFA Rules not processed for user `884aac2daddd438b96d16553e8805fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:24.064700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-456c44a4-2df8-45b6-8095-2e9d1ad6e598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:24.129468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1032/2084] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:56:24.140956 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476e7a2-0069-4527-8b5f-05ed17e882e4 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:24.142619 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a476e7a2-0069-4527-8b5f-05ed17e882e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:24.243089 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476e7a2-0069-4527-8b5f-05ed17e882e4 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:24.248808 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31823ac9-b734-4350-a79a-72412e6d064e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:24.249406 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31823ac9-b734-4350-a79a-72412e6d064e service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:24.249707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31823ac9-b734-4350-a79a-72412e6d064e service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:24.249975 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31823ac9-b734-4350-a79a-72412e6d064e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:24.258932 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31823ac9-b734-4350-a79a-72412e6d064e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:24.260527 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31823ac9-b734-4350-a79a-72412e6d064e service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:24.269607 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1053/2085] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:56:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:56:24.488387 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c44a4-2df8-45b6-8095-2e9d1ad6e598 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:24.551805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c44a4-2df8-45b6-8095-2e9d1ad6e598 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:24.556316 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8579a9d-9d3e-4bf1-b35e-dc1493da0ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:24.556936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8579a9d-9d3e-4bf1-b35e-dc1493da0ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:24.557211 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8579a9d-9d3e-4bf1-b35e-dc1493da0ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:24.557482 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8579a9d-9d3e-4bf1-b35e-dc1493da0ec6 admin admin] PATH_INFO: `/v3/users/884aac2daddd438b96d16553e8805fd9` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:24.558579 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8579a9d-9d3e-4bf1-b35e-dc1493da0ec6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=884aac2daddd438b96d16553e8805fd9)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:24.560584 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8579a9d-9d3e-4bf1-b35e-dc1493da0ec6 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:24.565117 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:24.607672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a8579a9d-9d3e-4bf1-b35e-dc1493da0ec6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '884aac2daddd438b96d16553e8805fd9'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:24.608351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a8579a9d-9d3e-4bf1-b35e-dc1493da0ec6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '884aac2daddd438b96d16553e8805fd9'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:24.621889 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a8579a9d-9d3e-4bf1-b35e-dc1493da0ec6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '884aac2daddd438b96d16553e8805fd9'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:24.628735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a8579a9d-9d3e-4bf1-b35e-dc1493da0ec6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '884aac2daddd438b96d16553e8805fd9'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:24.629381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a8579a9d-9d3e-4bf1-b35e-dc1493da0ec6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '884aac2daddd438b96d16553e8805fd9'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:24.643830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1033/2086] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:56:24 2022] DELETE /identity/v3/users/884aac2daddd438b96d16553e8805fd9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:24.648307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01acfb-7627-4d9f-939f-b7389146ac35 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:24.648791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01acfb-7627-4d9f-939f-b7389146ac35 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:24.648888 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01acfb-7627-4d9f-939f-b7389146ac35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:24.655970 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:24.674227 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-fa01acfb-7627-4d9f-939f-b7389146ac35 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:24.675980 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-fa01acfb-7627-4d9f-939f-b7389146ac35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:24.736315 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1054/2087] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:24.746932 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8579a9d-9d3e-4bf1-b35e-dc1493da0ec6 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:24.748408 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a8579a9d-9d3e-4bf1-b35e-dc1493da0ec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:24.805974 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8579a9d-9d3e-4bf1-b35e-dc1493da0ec6 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:24.809580 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975950d-2e60-410e-bf98-8f7c5eae39ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:24.810184 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975950d-2e60-410e-bf98-8f7c5eae39ea service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:24.810460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975950d-2e60-410e-bf98-8f7c5eae39ea service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:24.810730 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975950d-2e60-410e-bf98-8f7c5eae39ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:24.820204 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975950d-2e60-410e-bf98-8f7c5eae39ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:24.821634 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975950d-2e60-410e-bf98-8f7c5eae39ea service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:24.831681 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1034/2088] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:56:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:56:25.103220 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01acfb-7627-4d9f-939f-b7389146ac35 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:25.104690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-fa01acfb-7627-4d9f-939f-b7389146ac35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:25.119566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01acfb-7627-4d9f-939f-b7389146ac35 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:25.124275 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4054f-93a2-46d6-866a-bb0d70e3a076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:25.124908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4054f-93a2-46d6-866a-bb0d70e3a076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:25.125187 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4054f-93a2-46d6-866a-bb0d70e3a076 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:25.125457 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4054f-93a2-46d6-866a-bb0d70e3a076 admin admin] PATH_INFO: `/v3/projects/c9d7c91873be42cfbad24838de99e145` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:25.126603 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4054f-93a2-46d6-866a-bb0d70e3a076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9d7c91873be42cfbad24838de99e145)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:25.128819 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4054f-93a2-46d6-866a-bb0d70e3a076 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:25.176477 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-c8b4054f-93a2-46d6-866a-bb0d70e3a076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9d7c91873be42cfbad24838de99e145', 'request_id': 'req-c8b4054f-93a2-46d6-866a-bb0d70e3a076'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:25.183715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-c8b4054f-93a2-46d6-866a-bb0d70e3a076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9d7c91873be42cfbad24838de99e145', 'request_id': 'req-c8b4054f-93a2-46d6-866a-bb0d70e3a076'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:25.189355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-c8b4054f-93a2-46d6-866a-bb0d70e3a076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9d7c91873be42cfbad24838de99e145', 'request_id': 'req-c8b4054f-93a2-46d6-866a-bb0d70e3a076'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:25.200690 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-c8b4054f-93a2-46d6-866a-bb0d70e3a076 admin admin] The token cache is being invalidate because project c9d7c91873be42cfbad24838de99e145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:56:25.200976 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-c8b4054f-93a2-46d6-866a-bb0d70e3a076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:25.201635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-c8b4054f-93a2-46d6-866a-bb0d70e3a076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:25.203098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1055/2089] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:56:25 2022] DELETE /identity/v3/projects/c9d7c91873be42cfbad24838de99e145 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:25.208772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec93c9-7e46-4572-bc77-2b7b7e833523 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:25.209249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec93c9-7e46-4572-bc77-2b7b7e833523 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:25.209459 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec93c9-7e46-4572-bc77-2b7b7e833523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:25.229240 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-caec93c9-7e46-4572-bc77-2b7b7e833523 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:25.230429 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-caec93c9-7e46-4572-bc77-2b7b7e833523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:25.288738 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1035/2090] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:25.296674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fdcf9-d139-4bb8-ba25-f7680a0d8a8b None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:25.296988 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fdcf9-d139-4bb8-ba25-f7680a0d8a8b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:25.297268 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fdcf9-d139-4bb8-ba25-f7680a0d8a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:25.317527 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-266fdcf9-d139-4bb8-ba25-f7680a0d8a8b None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:25.318734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-266fdcf9-d139-4bb8-ba25-f7680a0d8a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:25.326581 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1056/2091] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:25.331913 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec93c9-7e46-4572-bc77-2b7b7e833523 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:25.342202 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec93c9-7e46-4572-bc77-2b7b7e833523 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:25.347239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5669027-60f9-4729-91e6-45f4018305b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:25.347859 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5669027-60f9-4729-91e6-45f4018305b7 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:25.348165 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5669027-60f9-4729-91e6-45f4018305b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:25.348438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5669027-60f9-4729-91e6-45f4018305b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:25.349079 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5669027-60f9-4729-91e6-45f4018305b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:56:25.349376 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5669027-60f9-4729-91e6-45f4018305b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:25.350980 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5669027-60f9-4729-91e6-45f4018305b7 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:25.362695 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1036/2092] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:56:25.367103 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fdcf9-d139-4bb8-ba25-f7680a0d8a8b None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:25.376332 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fdcf9-d139-4bb8-ba25-f7680a0d8a8b None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:25.381884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd52274-7fb8-47fb-bbba-826b31eb7312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:25.382502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd52274-7fb8-47fb-bbba-826b31eb7312 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:25.382782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd52274-7fb8-47fb-bbba-826b31eb7312 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:25.383056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd52274-7fb8-47fb-bbba-826b31eb7312 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:25.383697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd52274-7fb8-47fb-bbba-826b31eb7312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:25.385651 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd52274-7fb8-47fb-bbba-826b31eb7312 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:25.412991 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1057/2093] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:56:25 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:56:25.417189 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5669027-60f9-4729-91e6-45f4018305b7 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:25.426994 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5669027-60f9-4729-91e6-45f4018305b7 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:25.435573 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:25.486438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0911b0f-4c6c-481b-93a0-3a88caa8d364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:25.487150 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0911b0f-4c6c-481b-93a0-3a88caa8d364 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:25.487433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0911b0f-4c6c-481b-93a0-3a88caa8d364 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:25.487817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0911b0f-4c6c-481b-93a0-3a88caa8d364 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:25.488486 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0911b0f-4c6c-481b-93a0-3a88caa8d364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:25.490430 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0911b0f-4c6c-481b-93a0-3a88caa8d364 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:25.512437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1037/2094] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:56:25 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:56:25.516713 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd52274-7fb8-47fb-bbba-826b31eb7312 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:25.527825 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd52274-7fb8-47fb-bbba-826b31eb7312 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:25.532439 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab7fef-ce2e-40fd-997c-d15ceba009b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:25.533034 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab7fef-ce2e-40fd-997c-d15ceba009b3 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:25.533314 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab7fef-ce2e-40fd-997c-d15ceba009b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:25.533611 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab7fef-ce2e-40fd-997c-d15ceba009b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:25.534277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ab7fef-ce2e-40fd-997c-d15ceba009b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:25.535902 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ab7fef-ce2e-40fd-997c-d15ceba009b3 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:25.545762 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1058/2095] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:25.549609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0911b0f-4c6c-481b-93a0-3a88caa8d364 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:25.559748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0911b0f-4c6c-481b-93a0-3a88caa8d364 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:25.564352 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26846abe-e32b-41cd-89ca-3fdb778bc260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:25.565550 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26846abe-e32b-41cd-89ca-3fdb778bc260 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:25.565990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26846abe-e32b-41cd-89ca-3fdb778bc260 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:25.566278 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26846abe-e32b-41cd-89ca-3fdb778bc260 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:25.566942 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26846abe-e32b-41cd-89ca-3fdb778bc260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:25.568581 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26846abe-e32b-41cd-89ca-3fdb778bc260 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:25.579061 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1038/2096] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:25.583988 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab7fef-ce2e-40fd-997c-d15ceba009b3 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:25.593797 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab7fef-ce2e-40fd-997c-d15ceba009b3 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:25.598491 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4ebc5-1765-41b5-b3bc-ef6df35dbb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:25.599143 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4ebc5-1765-41b5-b3bc-ef6df35dbb14 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:25.599499 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4ebc5-1765-41b5-b3bc-ef6df35dbb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:25.600147 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4ebc5-1765-41b5-b3bc-ef6df35dbb14 admin admin] PATH_INFO: `/v3/projects/1f847ecec1314dc6babb64c48855b776/users/201e4950518a435594ff44b89e155a55/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:25.614607 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4ebc5-1765-41b5-b3bc-ef6df35dbb14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f847ecec1314dc6babb64c48855b776, user_id=201e4950518a435594ff44b89e155a55, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:25.617301 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4ebc5-1765-41b5-b3bc-ef6df35dbb14 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:25.630637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1059/2097] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:25 2022] PUT /identity/v3/projects/1f847ecec1314dc6babb64c48855b776/users/201e4950518a435594ff44b89e155a55/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:25.642193 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163b767-9a9d-4b6f-9509-ca2f7bf67132 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:25.642477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163b767-9a9d-4b6f-9509-ca2f7bf67132 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:25.642776 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163b767-9a9d-4b6f-9509-ca2f7bf67132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:25.670493 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-6163b767-9a9d-4b6f-9509-ca2f7bf67132 None None] MFA Rules not processed for user `201e4950518a435594ff44b89e155a55`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:25.671694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-6163b767-9a9d-4b6f-9509-ca2f7bf67132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:25.734432 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1039/2098] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:25.739901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4ebc5-1765-41b5-b3bc-ef6df35dbb14 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:25.766349 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:25.805204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4ebc5-1765-41b5-b3bc-ef6df35dbb14 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:25.809828 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07d18e-e65d-4c93-b0cc-b8941e9237af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:25.810430 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07d18e-e65d-4c93-b0cc-b8941e9237af admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:25.810707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07d18e-e65d-4c93-b0cc-b8941e9237af admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:25.810979 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07d18e-e65d-4c93-b0cc-b8941e9237af admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:25.811610 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07d18e-e65d-4c93-b0cc-b8941e9237af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:25.813539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07d18e-e65d-4c93-b0cc-b8941e9237af admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:25.840761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1060/2099] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:56:25 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:25.844784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163b767-9a9d-4b6f-9509-ca2f7bf67132 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:25.855984 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163b767-9a9d-4b6f-9509-ca2f7bf67132 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:25.913593 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37160e8-6dee-46a1-a0ad-740461156e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:25.914198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37160e8-6dee-46a1-a0ad-740461156e9c admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:25.914477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37160e8-6dee-46a1-a0ad-740461156e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:25.914751 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37160e8-6dee-46a1-a0ad-740461156e9c admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:25.915379 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37160e8-6dee-46a1-a0ad-740461156e9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:25.917333 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37160e8-6dee-46a1-a0ad-740461156e9c admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:25.940672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1040/2100] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:56:25 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:25.945073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07d18e-e65d-4c93-b0cc-b8941e9237af admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:25.955287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07d18e-e65d-4c93-b0cc-b8941e9237af admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:25.960585 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df58036-58b6-4688-b11b-c1fce53525d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:25.961200 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df58036-58b6-4688-b11b-c1fce53525d4 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:25.961483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df58036-58b6-4688-b11b-c1fce53525d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:25.961760 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df58036-58b6-4688-b11b-c1fce53525d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:25.962401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df58036-58b6-4688-b11b-c1fce53525d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:25.964037 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df58036-58b6-4688-b11b-c1fce53525d4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:25.974946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1061/2101] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:25.978958 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37160e8-6dee-46a1-a0ad-740461156e9c admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:25.989953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37160e8-6dee-46a1-a0ad-740461156e9c admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:25.994397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e0302-61a3-46ce-ae70-5e0639777569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:25.995036 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e0302-61a3-46ce-ae70-5e0639777569 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:25.995313 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e0302-61a3-46ce-ae70-5e0639777569 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:25.995582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e0302-61a3-46ce-ae70-5e0639777569 admin admin] PATH_INFO: `/v3/projects/8a9adf9f30da48ebad63fa11c18d96dc/users/c988502604854b25afa8b6940d209aac/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:26.005210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:26.012315 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e0302-61a3-46ce-ae70-5e0639777569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9adf9f30da48ebad63fa11c18d96dc, user_id=c988502604854b25afa8b6940d209aac, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:26.015297 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e0302-61a3-46ce-ae70-5e0639777569 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:26.028001 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1041/2102] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:25 2022] PUT /identity/v3/projects/8a9adf9f30da48ebad63fa11c18d96dc/users/c988502604854b25afa8b6940d209aac/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:26.031995 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df58036-58b6-4688-b11b-c1fce53525d4 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:26.042428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df58036-58b6-4688-b11b-c1fce53525d4 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:26.099541 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ab416-ba7c-4234-b9ad-9371eb5a1e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:26.100198 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ab416-ba7c-4234-b9ad-9371eb5a1e67 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:26.100500 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ab416-ba7c-4234-b9ad-9371eb5a1e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:26.100763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ab416-ba7c-4234-b9ad-9371eb5a1e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:26.101402 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ab416-ba7c-4234-b9ad-9371eb5a1e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:26.103047 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ab416-ba7c-4234-b9ad-9371eb5a1e67 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:26.113667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1062/2103] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:26.117988 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e0302-61a3-46ce-ae70-5e0639777569 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:26.128628 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e0302-61a3-46ce-ae70-5e0639777569 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:26.133119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e81a7-1aa8-4185-9a69-8b37a5d2370a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:26.133755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e81a7-1aa8-4185-9a69-8b37a5d2370a admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:26.134032 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e81a7-1aa8-4185-9a69-8b37a5d2370a admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:26.134303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e81a7-1aa8-4185-9a69-8b37a5d2370a admin admin] PATH_INFO: `/v3/projects/8a9adf9f30da48ebad63fa11c18d96dc/users/c988502604854b25afa8b6940d209aac/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:26.136203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e81a7-1aa8-4185-9a69-8b37a5d2370a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9adf9f30da48ebad63fa11c18d96dc, user_id=c988502604854b25afa8b6940d209aac, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:26.138865 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e81a7-1aa8-4185-9a69-8b37a5d2370a admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:26.149440 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1042/2104] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:26 2022] PUT /identity/v3/projects/8a9adf9f30da48ebad63fa11c18d96dc/users/c988502604854b25afa8b6940d209aac/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:26.160004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f20f7f-c26c-481e-8c8b-17bc38c47296 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:26.160354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f20f7f-c26c-481e-8c8b-17bc38c47296 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:26.160633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f20f7f-c26c-481e-8c8b-17bc38c47296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:26.188865 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-86f20f7f-c26c-481e-8c8b-17bc38c47296 None None] MFA Rules not processed for user `c988502604854b25afa8b6940d209aac`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:26.190133 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-86f20f7f-c26c-481e-8c8b-17bc38c47296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:26.259886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1063/2105] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:26.271588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e81a7-1aa8-4185-9a69-8b37a5d2370a admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:26.273390 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-066e81a7-1aa8-4185-9a69-8b37a5d2370a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:26.330316 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e81a7-1aa8-4185-9a69-8b37a5d2370a admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:26.334552 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d04b8d-0480-4d33-841d-c7347b213840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:26.335153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d04b8d-0480-4d33-841d-c7347b213840 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:26.335434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d04b8d-0480-4d33-841d-c7347b213840 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:26.335710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d04b8d-0480-4d33-841d-c7347b213840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:26.344831 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d04b8d-0480-4d33-841d-c7347b213840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:26.346917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d04b8d-0480-4d33-841d-c7347b213840 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:26.356675 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1043/2106] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:56:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:56:26.432369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f20f7f-c26c-481e-8c8b-17bc38c47296 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:26.442204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f20f7f-c26c-481e-8c8b-17bc38c47296 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:26.445779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3746fad-f510-49d7-a25e-5f8e0decc751 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:26.446376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3746fad-f510-49d7-a25e-5f8e0decc751 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:26.446651 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3746fad-f510-49d7-a25e-5f8e0decc751 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:26.446920 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3746fad-f510-49d7-a25e-5f8e0decc751 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:26.460025 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:26.505254 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3746fad-f510-49d7-a25e-5f8e0decc751 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:26.506695 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3746fad-f510-49d7-a25e-5f8e0decc751 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:26.515567 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1064/2107] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:56:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:56:36.205987 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d04b8d-0480-4d33-841d-c7347b213840 service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:36.208015 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-95d04b8d-0480-4d33-841d-c7347b213840 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:36.321870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d04b8d-0480-4d33-841d-c7347b213840 service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:36.326845 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4967673-61df-43f2-8c4c-11ff865bd72e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:36.327451 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4967673-61df-43f2-8c4c-11ff865bd72e service placement] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:36.327727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4967673-61df-43f2-8c4c-11ff865bd72e service placement] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:36.327998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4967673-61df-43f2-8c4c-11ff865bd72e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:36.329728 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c4967673-61df-43f2-8c4c-11ff865bd72e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:36.341626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4967673-61df-43f2-8c4c-11ff865bd72e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:36.343183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4967673-61df-43f2-8c4c-11ff865bd72e service placement] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:36.353492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1044/2108] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:56:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:56:36.511484 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=20 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:37.842997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3746fad-f510-49d7-a25e-5f8e0decc751 service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:37.915188 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3746fad-f510-49d7-a25e-5f8e0decc751 service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:37.923131 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6a7cc-58cd-4a16-aa30-e8a0c737cae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:37.923895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6a7cc-58cd-4a16-aa30-e8a0c737cae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:37.924304 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6a7cc-58cd-4a16-aa30-e8a0c737cae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:37.924676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6a7cc-58cd-4a16-aa30-e8a0c737cae1 admin admin] PATH_INFO: `/v3/users/201e4950518a435594ff44b89e155a55` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:37.927077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6a7cc-58cd-4a16-aa30-e8a0c737cae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=201e4950518a435594ff44b89e155a55)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:37.929096 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6a7cc-58cd-4a16-aa30-e8a0c737cae1 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:37.980902 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f9b6a7cc-58cd-4a16-aa30-e8a0c737cae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '201e4950518a435594ff44b89e155a55'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:37.981595 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f9b6a7cc-58cd-4a16-aa30-e8a0c737cae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '201e4950518a435594ff44b89e155a55'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:37.995967 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f9b6a7cc-58cd-4a16-aa30-e8a0c737cae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '201e4950518a435594ff44b89e155a55'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:38.003396 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f9b6a7cc-58cd-4a16-aa30-e8a0c737cae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '201e4950518a435594ff44b89e155a55'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:38.012924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f9b6a7cc-58cd-4a16-aa30-e8a0c737cae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '201e4950518a435594ff44b89e155a55'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:38.019236 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1065/2109] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:56:37 2022] DELETE /identity/v3/users/201e4950518a435594ff44b89e155a55 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:38.023710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6175f54-93e6-4fa0-a931-82e6e665fb77 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:38.024241 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6175f54-93e6-4fa0-a931-82e6e665fb77 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:38.024422 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6175f54-93e6-4fa0-a931-82e6e665fb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:38.045502 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-d6175f54-93e6-4fa0-a931-82e6e665fb77 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:38.046705 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d6175f54-93e6-4fa0-a931-82e6e665fb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:38.097320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:38.117469 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1045/2110] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:38.127037 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6a7cc-58cd-4a16-aa30-e8a0c737cae1 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:38.128666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f9b6a7cc-58cd-4a16-aa30-e8a0c737cae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:38.213349 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6a7cc-58cd-4a16-aa30-e8a0c737cae1 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:38.217740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d174a1-a657-4edd-952a-e2bbc3daee1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:38.218372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d174a1-a657-4edd-952a-e2bbc3daee1c service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:38.218654 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d174a1-a657-4edd-952a-e2bbc3daee1c service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:38.218931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d174a1-a657-4edd-952a-e2bbc3daee1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:38.230040 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d174a1-a657-4edd-952a-e2bbc3daee1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:38.231477 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d174a1-a657-4edd-952a-e2bbc3daee1c service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:38.242122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1066/2111] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:56:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:56:38.938677 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6175f54-93e6-4fa0-a931-82e6e665fb77 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:38.940301 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d6175f54-93e6-4fa0-a931-82e6e665fb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:38.952732 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6175f54-93e6-4fa0-a931-82e6e665fb77 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:38.957640 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b5f2e-b8cd-48df-be61-9401f64d7331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:38.958303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b5f2e-b8cd-48df-be61-9401f64d7331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:38.958624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b5f2e-b8cd-48df-be61-9401f64d7331 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:38.958938 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b5f2e-b8cd-48df-be61-9401f64d7331 admin admin] PATH_INFO: `/v3/projects/1f847ecec1314dc6babb64c48855b776` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:38.961152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b5f2e-b8cd-48df-be61-9401f64d7331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f847ecec1314dc6babb64c48855b776)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:38.963247 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b5f2e-b8cd-48df-be61-9401f64d7331 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:39.018606 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e47b5f2e-b8cd-48df-be61-9401f64d7331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f847ecec1314dc6babb64c48855b776', 'request_id': 'req-e47b5f2e-b8cd-48df-be61-9401f64d7331'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:39.027899 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e47b5f2e-b8cd-48df-be61-9401f64d7331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f847ecec1314dc6babb64c48855b776', 'request_id': 'req-e47b5f2e-b8cd-48df-be61-9401f64d7331'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:39.034626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e47b5f2e-b8cd-48df-be61-9401f64d7331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f847ecec1314dc6babb64c48855b776', 'request_id': 'req-e47b5f2e-b8cd-48df-be61-9401f64d7331'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:39.046819 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e47b5f2e-b8cd-48df-be61-9401f64d7331 admin admin] The token cache is being invalidate because project 1f847ecec1314dc6babb64c48855b776 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:56:39.047108 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e47b5f2e-b8cd-48df-be61-9401f64d7331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:39.047794 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e47b5f2e-b8cd-48df-be61-9401f64d7331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:39.049767 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1046/2112] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:56:38 2022] DELETE /identity/v3/projects/1f847ecec1314dc6babb64c48855b776 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:39.055210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d174a1-a657-4edd-952a-e2bbc3daee1c service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:39.060887 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:39.067657 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d174a1-a657-4edd-952a-e2bbc3daee1c service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:39.127978 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be577c52-0631-4082-a134-f1a20d1fcc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:39.129658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be577c52-0631-4082-a134-f1a20d1fcc14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:39.129658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be577c52-0631-4082-a134-f1a20d1fcc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:39.130085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be577c52-0631-4082-a134-f1a20d1fcc14 admin admin] PATH_INFO: `/v3/users/c988502604854b25afa8b6940d209aac` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:39.131248 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be577c52-0631-4082-a134-f1a20d1fcc14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c988502604854b25afa8b6940d209aac)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:39.133285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be577c52-0631-4082-a134-f1a20d1fcc14 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:39.182609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-be577c52-0631-4082-a134-f1a20d1fcc14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c988502604854b25afa8b6940d209aac'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:39.183413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-be577c52-0631-4082-a134-f1a20d1fcc14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c988502604854b25afa8b6940d209aac'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:39.197508 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-be577c52-0631-4082-a134-f1a20d1fcc14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c988502604854b25afa8b6940d209aac'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:39.204708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-be577c52-0631-4082-a134-f1a20d1fcc14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c988502604854b25afa8b6940d209aac'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:39.215536 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-be577c52-0631-4082-a134-f1a20d1fcc14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c988502604854b25afa8b6940d209aac'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:39.222108 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1067/2113] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:56:39 2022] DELETE /identity/v3/users/c988502604854b25afa8b6940d209aac => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:39.513630 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b5f2e-b8cd-48df-be61-9401f64d7331 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:39.516385 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e47b5f2e-b8cd-48df-be61-9401f64d7331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:39.546917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b5f2e-b8cd-48df-be61-9401f64d7331 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:39.604661 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7dff1-66d8-4ba0-94e6-92ff27ec9cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:39.605304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7dff1-66d8-4ba0-94e6-92ff27ec9cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:39.605737 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7dff1-66d8-4ba0-94e6-92ff27ec9cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:39.606015 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7dff1-66d8-4ba0-94e6-92ff27ec9cd7 admin admin] PATH_INFO: `/v3/projects/8a9adf9f30da48ebad63fa11c18d96dc` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:39.607254 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7dff1-66d8-4ba0-94e6-92ff27ec9cd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a9adf9f30da48ebad63fa11c18d96dc)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:39.609244 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7dff1-66d8-4ba0-94e6-92ff27ec9cd7 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:39.639463 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5aac1-8faa-47a4-bf56-b08cdbfdac60 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:39.639736 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5aac1-8faa-47a4-bf56-b08cdbfdac60 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:39.640112 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5aac1-8faa-47a4-bf56-b08cdbfdac60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:39.660339 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6fc7dff1-66d8-4ba0-94e6-92ff27ec9cd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a9adf9f30da48ebad63fa11c18d96dc', 'request_id': 'req-6fc7dff1-66d8-4ba0-94e6-92ff27ec9cd7'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:39.661992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-4ea5aac1-8faa-47a4-bf56-b08cdbfdac60 None None] MFA Rules not processed for user `5ac3fdb86ac24fa493706f84e1aa40a8`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:39.663261 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4ea5aac1-8faa-47a4-bf56-b08cdbfdac60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:39.669590 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6fc7dff1-66d8-4ba0-94e6-92ff27ec9cd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a9adf9f30da48ebad63fa11c18d96dc', 'request_id': 'req-6fc7dff1-66d8-4ba0-94e6-92ff27ec9cd7'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:39.676310 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6fc7dff1-66d8-4ba0-94e6-92ff27ec9cd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a9adf9f30da48ebad63fa11c18d96dc', 'request_id': 'req-6fc7dff1-66d8-4ba0-94e6-92ff27ec9cd7'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:39.685619 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:39.689684 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6fc7dff1-66d8-4ba0-94e6-92ff27ec9cd7 admin admin] The token cache is being invalidate because project 8a9adf9f30da48ebad63fa11c18d96dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:56:39.690118 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6fc7dff1-66d8-4ba0-94e6-92ff27ec9cd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:39.691062 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6fc7dff1-66d8-4ba0-94e6-92ff27ec9cd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:39.693726 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1047/2114] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:56:39 2022] DELETE /identity/v3/projects/8a9adf9f30da48ebad63fa11c18d96dc => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:39.698301 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4c539-d22c-472e-ad24-55fab8c47f54 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:39.698617 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4c539-d22c-472e-ad24-55fab8c47f54 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:39.698893 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4c539-d22c-472e-ad24-55fab8c47f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:39.719923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-43f4c539-d22c-472e-ad24-55fab8c47f54 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:39.721222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-43f4c539-d22c-472e-ad24-55fab8c47f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:39.721793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1068/2115] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:56:39.788273 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1048/2116] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:56:39.797421 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004c6b0-1908-4881-acbf-ca1520609811 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:39.797703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004c6b0-1908-4881-acbf-ca1520609811 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:39.797977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004c6b0-1908-4881-acbf-ca1520609811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:39.819316 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-c004c6b0-1908-4881-acbf-ca1520609811 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:39.820562 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c004c6b0-1908-4881-acbf-ca1520609811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:39.828591 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1069/2117] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:39.833847 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4c539-d22c-472e-ad24-55fab8c47f54 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:39.843313 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4c539-d22c-472e-ad24-55fab8c47f54 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:39.847963 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871cbdf-123d-4737-8578-9b4091a91b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:39.848594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871cbdf-123d-4737-8578-9b4091a91b81 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:39.848879 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871cbdf-123d-4737-8578-9b4091a91b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:39.849149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871cbdf-123d-4737-8578-9b4091a91b81 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:39.849790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871cbdf-123d-4737-8578-9b4091a91b81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:56:39.850090 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871cbdf-123d-4737-8578-9b4091a91b81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:39.851687 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871cbdf-123d-4737-8578-9b4091a91b81 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:39.864590 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1049/2118] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:56:39.868990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004c6b0-1908-4881-acbf-ca1520609811 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:39.880340 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004c6b0-1908-4881-acbf-ca1520609811 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:39.884985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadafde-268d-4b2b-be6a-c8cb42e66141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:39.885671 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadafde-268d-4b2b-be6a-c8cb42e66141 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:39.885953 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadafde-268d-4b2b-be6a-c8cb42e66141 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:39.886224 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadafde-268d-4b2b-be6a-c8cb42e66141 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:39.886860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadafde-268d-4b2b-be6a-c8cb42e66141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:39.888825 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadafde-268d-4b2b-be6a-c8cb42e66141 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:39.912546 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:39.921225 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1070/2119] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:56:39 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:39.926753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871cbdf-123d-4737-8578-9b4091a91b81 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:39.937157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871cbdf-123d-4737-8578-9b4091a91b81 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:39.999462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2f395-f35c-4ec6-b90a-2e53c31965dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:40.000077 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2f395-f35c-4ec6-b90a-2e53c31965dc admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:40.000382 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2f395-f35c-4ec6-b90a-2e53c31965dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:40.000655 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2f395-f35c-4ec6-b90a-2e53c31965dc admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:40.001291 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2f395-f35c-4ec6-b90a-2e53c31965dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:40.003226 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2f395-f35c-4ec6-b90a-2e53c31965dc admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:40.028688 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1050/2120] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:56:39 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:40.029910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadafde-268d-4b2b-be6a-c8cb42e66141 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:40.047429 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadafde-268d-4b2b-be6a-c8cb42e66141 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:40.052998 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bba219-35f6-4ae7-a4bc-fcaf5f6e67a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:40.053708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bba219-35f6-4ae7-a4bc-fcaf5f6e67a6 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:40.054164 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bba219-35f6-4ae7-a4bc-fcaf5f6e67a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:40.054508 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bba219-35f6-4ae7-a4bc-fcaf5f6e67a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:40.055224 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bba219-35f6-4ae7-a4bc-fcaf5f6e67a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:40.056899 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bba219-35f6-4ae7-a4bc-fcaf5f6e67a6 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:40.071663 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1071/2121] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:56:40.072974 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2f395-f35c-4ec6-b90a-2e53c31965dc admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:40.084263 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2f395-f35c-4ec6-b90a-2e53c31965dc admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:40.089052 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedeff90-6c5f-49c6-b109-9d572dc52b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:40.089655 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedeff90-6c5f-49c6-b109-9d572dc52b18 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:40.089936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedeff90-6c5f-49c6-b109-9d572dc52b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:40.090217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedeff90-6c5f-49c6-b109-9d572dc52b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:40.090855 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedeff90-6c5f-49c6-b109-9d572dc52b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:40.092503 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedeff90-6c5f-49c6-b109-9d572dc52b18 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:40.103599 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1051/2122] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:40.107844 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bba219-35f6-4ae7-a4bc-fcaf5f6e67a6 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:40.118741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bba219-35f6-4ae7-a4bc-fcaf5f6e67a6 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:40.123631 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdec94-ee63-499b-96fe-a4b5be9bb6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:40.124422 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdec94-ee63-499b-96fe-a4b5be9bb6a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:40.124773 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdec94-ee63-499b-96fe-a4b5be9bb6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:40.125113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdec94-ee63-499b-96fe-a4b5be9bb6a2 admin admin] PATH_INFO: `/v3/projects/380c79bc591e4c838ccec570374c52bd/users/10c9b6c2e84242d9aaeffa1cca9a9dd4/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:40.141751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bdec94-ee63-499b-96fe-a4b5be9bb6a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=380c79bc591e4c838ccec570374c52bd, user_id=10c9b6c2e84242d9aaeffa1cca9a9dd4, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:40.144731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bdec94-ee63-499b-96fe-a4b5be9bb6a2 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:40.158410 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1072/2123] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:40 2022] PUT /identity/v3/projects/380c79bc591e4c838ccec570374c52bd/users/10c9b6c2e84242d9aaeffa1cca9a9dd4/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:56:40.168971 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3c002-1875-4e2c-8ea7-e82be43bf4ba None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:40.169240 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3c002-1875-4e2c-8ea7-e82be43bf4ba None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:40.169518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3c002-1875-4e2c-8ea7-e82be43bf4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:40.200868 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-f1a3c002-1875-4e2c-8ea7-e82be43bf4ba None None] MFA Rules not processed for user `10c9b6c2e84242d9aaeffa1cca9a9dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:40.202100 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-f1a3c002-1875-4e2c-8ea7-e82be43bf4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:40.262164 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:40.272897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1052/2124] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:40.277532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdec94-ee63-499b-96fe-a4b5be9bb6a2 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:40.352923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdec94-ee63-499b-96fe-a4b5be9bb6a2 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:40.356154 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4195c8-9570-4130-a241-0a79e4574760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:40.356777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4195c8-9570-4130-a241-0a79e4574760 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:40.357060 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4195c8-9570-4130-a241-0a79e4574760 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:40.357495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4195c8-9570-4130-a241-0a79e4574760 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:40.358505 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4195c8-9570-4130-a241-0a79e4574760 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:40.360045 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4195c8-9570-4130-a241-0a79e4574760 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:40.388487 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1073/2125] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:56:40 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:56:40.392648 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3c002-1875-4e2c-8ea7-e82be43bf4ba None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:40.404540 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3c002-1875-4e2c-8ea7-e82be43bf4ba None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:40.464006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98835b0f-6d59-4d4b-b8fe-5864344d07ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:40.464688 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98835b0f-6d59-4d4b-b8fe-5864344d07ba admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:40.464998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98835b0f-6d59-4d4b-b8fe-5864344d07ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:40.465300 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98835b0f-6d59-4d4b-b8fe-5864344d07ba admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:40.466029 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98835b0f-6d59-4d4b-b8fe-5864344d07ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:40.468149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98835b0f-6d59-4d4b-b8fe-5864344d07ba admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:40.490438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1053/2126] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:56:40 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:56:40.494734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4195c8-9570-4130-a241-0a79e4574760 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:40.505431 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4195c8-9570-4130-a241-0a79e4574760 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:40.510069 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24222e8f-d81c-4c1c-9aa4-56b9b83d27b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:40.510665 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24222e8f-d81c-4c1c-9aa4-56b9b83d27b3 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:40.510943 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24222e8f-d81c-4c1c-9aa4-56b9b83d27b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:40.511216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24222e8f-d81c-4c1c-9aa4-56b9b83d27b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:40.511846 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24222e8f-d81c-4c1c-9aa4-56b9b83d27b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:40.513485 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24222e8f-d81c-4c1c-9aa4-56b9b83d27b3 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:40.524119 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1074/2127] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:40.528174 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98835b0f-6d59-4d4b-b8fe-5864344d07ba admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:40.539070 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98835b0f-6d59-4d4b-b8fe-5864344d07ba admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:40.543647 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c10535-b25f-4a9e-87ac-0388fec33d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:40.544316 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c10535-b25f-4a9e-87ac-0388fec33d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:40.544593 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c10535-b25f-4a9e-87ac-0388fec33d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:40.544861 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c10535-b25f-4a9e-87ac-0388fec33d60 admin admin] PATH_INFO: `/v3/projects/162d75d74f12422197ac1cd4a124f6f3/users/4e0f3b7ab3b140bead7abdfc10870507/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:40.560836 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c10535-b25f-4a9e-87ac-0388fec33d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=162d75d74f12422197ac1cd4a124f6f3, user_id=4e0f3b7ab3b140bead7abdfc10870507, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:40.563500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c10535-b25f-4a9e-87ac-0388fec33d60 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:40.576898 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1054/2128] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:40 2022] PUT /identity/v3/projects/162d75d74f12422197ac1cd4a124f6f3/users/4e0f3b7ab3b140bead7abdfc10870507/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:40.580767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24222e8f-d81c-4c1c-9aa4-56b9b83d27b3 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:40.590927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24222e8f-d81c-4c1c-9aa4-56b9b83d27b3 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:40.648558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e86c4f-6e7e-4798-b647-a1310916f12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:40.649156 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e86c4f-6e7e-4798-b647-a1310916f12e admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:40.649432 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e86c4f-6e7e-4798-b647-a1310916f12e admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:40.649701 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e86c4f-6e7e-4798-b647-a1310916f12e admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:40.650322 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e86c4f-6e7e-4798-b647-a1310916f12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:40.651906 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e86c4f-6e7e-4798-b647-a1310916f12e admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:40.662343 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1075/2129] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:40.666457 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c10535-b25f-4a9e-87ac-0388fec33d60 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:40.676857 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c10535-b25f-4a9e-87ac-0388fec33d60 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:40.681395 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2f816-0279-4a63-8559-a25b062da544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:40.682029 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2f816-0279-4a63-8559-a25b062da544 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:40.682308 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2f816-0279-4a63-8559-a25b062da544 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:40.682575 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2f816-0279-4a63-8559-a25b062da544 admin admin] PATH_INFO: `/v3/projects/162d75d74f12422197ac1cd4a124f6f3/users/4e0f3b7ab3b140bead7abdfc10870507/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:40.684474 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2f816-0279-4a63-8559-a25b062da544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=162d75d74f12422197ac1cd4a124f6f3, user_id=4e0f3b7ab3b140bead7abdfc10870507, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:40.687104 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2f816-0279-4a63-8559-a25b062da544 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:40.697096 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1055/2130] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:40 2022] PUT /identity/v3/projects/162d75d74f12422197ac1cd4a124f6f3/users/4e0f3b7ab3b140bead7abdfc10870507/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:40.707848 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c6ced-05dd-42d5-874c-0aef8a3787e9 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:40.708165 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c6ced-05dd-42d5-874c-0aef8a3787e9 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:40.708467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c6ced-05dd-42d5-874c-0aef8a3787e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:40.736531 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-eb9c6ced-05dd-42d5-874c-0aef8a3787e9 None None] MFA Rules not processed for user `4e0f3b7ab3b140bead7abdfc10870507`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:40.737742 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-eb9c6ced-05dd-42d5-874c-0aef8a3787e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:40.757315 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:40.806497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1076/2131] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:40.820233 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2f816-0279-4a63-8559-a25b062da544 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:40.821956 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-80f2f816-0279-4a63-8559-a25b062da544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:40.904109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2f816-0279-4a63-8559-a25b062da544 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:40.907705 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d84de-dda4-42ec-be8e-d16f7daa6b39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:40.908333 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d84de-dda4-42ec-be8e-d16f7daa6b39 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:40.908613 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d84de-dda4-42ec-be8e-d16f7daa6b39 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:40.908886 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d84de-dda4-42ec-be8e-d16f7daa6b39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:40.918714 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d84de-dda4-42ec-be8e-d16f7daa6b39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:40.920185 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d84de-dda4-42ec-be8e-d16f7daa6b39 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:40.929733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1056/2132] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:56:40 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:56:41.006389 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c6ced-05dd-42d5-874c-0aef8a3787e9 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:41.015887 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c6ced-05dd-42d5-874c-0aef8a3787e9 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:41.019436 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae266e-d69e-40d5-8187-b29b2fd41866 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:41.020182 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae266e-d69e-40d5-8187-b29b2fd41866 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:41.021025 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae266e-d69e-40d5-8187-b29b2fd41866 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:41.021291 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae266e-d69e-40d5-8187-b29b2fd41866 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:41.078317 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae266e-d69e-40d5-8187-b29b2fd41866 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:41.079754 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae266e-d69e-40d5-8187-b29b2fd41866 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:41.089671 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1077/2133] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:56:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:56:43.817866 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d84de-dda4-42ec-be8e-d16f7daa6b39 service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:43.851864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:43.881174 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d84de-dda4-42ec-be8e-d16f7daa6b39 service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:43.885972 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a370c1f-7157-4d29-a634-8c8034e71fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:43.886597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a370c1f-7157-4d29-a634-8c8034e71fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:43.886875 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a370c1f-7157-4d29-a634-8c8034e71fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:43.887153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a370c1f-7157-4d29-a634-8c8034e71fe2 admin admin] PATH_INFO: `/v3/users/10c9b6c2e84242d9aaeffa1cca9a9dd4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:43.888317 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a370c1f-7157-4d29-a634-8c8034e71fe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10c9b6c2e84242d9aaeffa1cca9a9dd4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:43.890253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a370c1f-7157-4d29-a634-8c8034e71fe2 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:43.938652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-0a370c1f-7157-4d29-a634-8c8034e71fe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10c9b6c2e84242d9aaeffa1cca9a9dd4'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:43.939300 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-0a370c1f-7157-4d29-a634-8c8034e71fe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10c9b6c2e84242d9aaeffa1cca9a9dd4'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:43.951408 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-0a370c1f-7157-4d29-a634-8c8034e71fe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10c9b6c2e84242d9aaeffa1cca9a9dd4'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:43.958354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-0a370c1f-7157-4d29-a634-8c8034e71fe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10c9b6c2e84242d9aaeffa1cca9a9dd4'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:43.958984 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-0a370c1f-7157-4d29-a634-8c8034e71fe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10c9b6c2e84242d9aaeffa1cca9a9dd4'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:43.972733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1057/2134] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:56:43 2022] DELETE /identity/v3/users/10c9b6c2e84242d9aaeffa1cca9a9dd4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:43.977618 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01c973-42fb-42db-81a3-0f01a61942e9 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:43.978102 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01c973-42fb-42db-81a3-0f01a61942e9 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:43.978199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01c973-42fb-42db-81a3-0f01a61942e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:43.997299 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-fb01c973-42fb-42db-81a3-0f01a61942e9 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:43.998548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-fb01c973-42fb-42db-81a3-0f01a61942e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:44.058398 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1078/2135] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:44.068070 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a370c1f-7157-4d29-a634-8c8034e71fe2 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:44.070006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-0a370c1f-7157-4d29-a634-8c8034e71fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:44.125801 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a370c1f-7157-4d29-a634-8c8034e71fe2 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:44.129383 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a30fbe-fece-49c6-8d28-799137f48e82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:44.129976 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a30fbe-fece-49c6-8d28-799137f48e82 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:44.130255 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a30fbe-fece-49c6-8d28-799137f48e82 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:44.130521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a30fbe-fece-49c6-8d28-799137f48e82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:44.140205 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a30fbe-fece-49c6-8d28-799137f48e82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:44.141584 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a30fbe-fece-49c6-8d28-799137f48e82 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:44.151428 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1058/2136] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:56:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:56:44.421797 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01c973-42fb-42db-81a3-0f01a61942e9 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:44.423240 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-fb01c973-42fb-42db-81a3-0f01a61942e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:44.435717 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01c973-42fb-42db-81a3-0f01a61942e9 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:44.440489 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd161081-e4d6-4905-b228-de60ffb4ec83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:44.441196 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd161081-e4d6-4905-b228-de60ffb4ec83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:44.441545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd161081-e4d6-4905-b228-de60ffb4ec83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:44.441910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd161081-e4d6-4905-b228-de60ffb4ec83 admin admin] PATH_INFO: `/v3/projects/380c79bc591e4c838ccec570374c52bd` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:44.443092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd161081-e4d6-4905-b228-de60ffb4ec83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=380c79bc591e4c838ccec570374c52bd)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:44.445958 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd161081-e4d6-4905-b228-de60ffb4ec83 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:44.493043 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-bd161081-e4d6-4905-b228-de60ffb4ec83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '380c79bc591e4c838ccec570374c52bd', 'request_id': 'req-bd161081-e4d6-4905-b228-de60ffb4ec83'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:44.495496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:44.501963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-bd161081-e4d6-4905-b228-de60ffb4ec83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '380c79bc591e4c838ccec570374c52bd', 'request_id': 'req-bd161081-e4d6-4905-b228-de60ffb4ec83'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:44.509133 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-bd161081-e4d6-4905-b228-de60ffb4ec83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '380c79bc591e4c838ccec570374c52bd', 'request_id': 'req-bd161081-e4d6-4905-b228-de60ffb4ec83'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:44.520052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-bd161081-e4d6-4905-b228-de60ffb4ec83 admin admin] The token cache is being invalidate because project 380c79bc591e4c838ccec570374c52bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:56:44.520383 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-bd161081-e4d6-4905-b228-de60ffb4ec83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:44.521158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-bd161081-e4d6-4905-b228-de60ffb4ec83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:44.522734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1079/2137] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:56:44 2022] DELETE /identity/v3/projects/380c79bc591e4c838ccec570374c52bd => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:44.526784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a30fbe-fece-49c6-8d28-799137f48e82 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:44.536623 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a30fbe-fece-49c6-8d28-799137f48e82 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:44.584722 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:44.594708 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c60ef9-9e52-40d0-96a6-f227a710db8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:44.595543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c60ef9-9e52-40d0-96a6-f227a710db8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:44.595825 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c60ef9-9e52-40d0-96a6-f227a710db8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:44.596126 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c60ef9-9e52-40d0-96a6-f227a710db8e admin admin] PATH_INFO: `/v3/users/4e0f3b7ab3b140bead7abdfc10870507` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:44.597242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c60ef9-9e52-40d0-96a6-f227a710db8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e0f3b7ab3b140bead7abdfc10870507)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:44.599274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c60ef9-9e52-40d0-96a6-f227a710db8e admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:44.645402 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-74c60ef9-9e52-40d0-96a6-f227a710db8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e0f3b7ab3b140bead7abdfc10870507'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:44.646085 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-74c60ef9-9e52-40d0-96a6-f227a710db8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e0f3b7ab3b140bead7abdfc10870507'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:44.657918 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-74c60ef9-9e52-40d0-96a6-f227a710db8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e0f3b7ab3b140bead7abdfc10870507'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:44.665148 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-74c60ef9-9e52-40d0-96a6-f227a710db8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e0f3b7ab3b140bead7abdfc10870507'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:44.665741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-74c60ef9-9e52-40d0-96a6-f227a710db8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e0f3b7ab3b140bead7abdfc10870507'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:44.677926 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1059/2138] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:56:44 2022] DELETE /identity/v3/users/4e0f3b7ab3b140bead7abdfc10870507 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:44.968019 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd161081-e4d6-4905-b228-de60ffb4ec83 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:44.970037 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-bd161081-e4d6-4905-b228-de60ffb4ec83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:44.985047 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd161081-e4d6-4905-b228-de60ffb4ec83 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:45.046874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96048f7d-4d3b-4ff9-bedb-d3cbb5a08a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:45.047581 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96048f7d-4d3b-4ff9-bedb-d3cbb5a08a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:45.047863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96048f7d-4d3b-4ff9-bedb-d3cbb5a08a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:45.048204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96048f7d-4d3b-4ff9-bedb-d3cbb5a08a98 admin admin] PATH_INFO: `/v3/projects/162d75d74f12422197ac1cd4a124f6f3` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:45.049547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96048f7d-4d3b-4ff9-bedb-d3cbb5a08a98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=162d75d74f12422197ac1cd4a124f6f3)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:45.051891 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96048f7d-4d3b-4ff9-bedb-d3cbb5a08a98 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:45.103134 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-96048f7d-4d3b-4ff9-bedb-d3cbb5a08a98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '162d75d74f12422197ac1cd4a124f6f3', 'request_id': 'req-96048f7d-4d3b-4ff9-bedb-d3cbb5a08a98'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:45.110902 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-96048f7d-4d3b-4ff9-bedb-d3cbb5a08a98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '162d75d74f12422197ac1cd4a124f6f3', 'request_id': 'req-96048f7d-4d3b-4ff9-bedb-d3cbb5a08a98'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:45.116805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-96048f7d-4d3b-4ff9-bedb-d3cbb5a08a98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '162d75d74f12422197ac1cd4a124f6f3', 'request_id': 'req-96048f7d-4d3b-4ff9-bedb-d3cbb5a08a98'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:45.129900 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-96048f7d-4d3b-4ff9-bedb-d3cbb5a08a98 admin admin] The token cache is being invalidate because project 162d75d74f12422197ac1cd4a124f6f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:56:45.130202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-96048f7d-4d3b-4ff9-bedb-d3cbb5a08a98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:45.130912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-96048f7d-4d3b-4ff9-bedb-d3cbb5a08a98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:45.132697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1080/2139] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:56:44 2022] DELETE /identity/v3/projects/162d75d74f12422197ac1cd4a124f6f3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:45.138520 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdd6d6-f2e5-4ac2-b67a-7339d3bde759 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:45.139001 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdd6d6-f2e5-4ac2-b67a-7339d3bde759 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:45.139112 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdd6d6-f2e5-4ac2-b67a-7339d3bde759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:45.159299 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-fabdd6d6-f2e5-4ac2-b67a-7339d3bde759 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:45.160553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-fabdd6d6-f2e5-4ac2-b67a-7339d3bde759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:45.220713 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1060/2140] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:45.230178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300cf212-89ca-4068-a46a-6f2170ada134 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:45.230465 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300cf212-89ca-4068-a46a-6f2170ada134 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:45.230772 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300cf212-89ca-4068-a46a-6f2170ada134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:45.251031 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-300cf212-89ca-4068-a46a-6f2170ada134 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:45.252278 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-300cf212-89ca-4068-a46a-6f2170ada134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:45.260647 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1081/2141] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:45.264821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdd6d6-f2e5-4ac2-b67a-7339d3bde759 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:45.274513 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdd6d6-f2e5-4ac2-b67a-7339d3bde759 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:45.279125 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02619f1-dc49-4a2c-8760-0c6c57d740f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:45.279722 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02619f1-dc49-4a2c-8760-0c6c57d740f0 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:45.280001 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02619f1-dc49-4a2c-8760-0c6c57d740f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:45.280304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02619f1-dc49-4a2c-8760-0c6c57d740f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:45.280952 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02619f1-dc49-4a2c-8760-0c6c57d740f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:56:45.281256 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02619f1-dc49-4a2c-8760-0c6c57d740f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:45.282862 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02619f1-dc49-4a2c-8760-0c6c57d740f0 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:45.294515 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1061/2142] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:56:45.298721 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300cf212-89ca-4068-a46a-6f2170ada134 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:45.308530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300cf212-89ca-4068-a46a-6f2170ada134 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:45.313160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555c83b-ade2-4358-b112-04cbc1cbedd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:45.313742 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555c83b-ade2-4358-b112-04cbc1cbedd4 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:45.314021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555c83b-ade2-4358-b112-04cbc1cbedd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:45.314295 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555c83b-ade2-4358-b112-04cbc1cbedd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:45.314931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555c83b-ade2-4358-b112-04cbc1cbedd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:45.316860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555c83b-ade2-4358-b112-04cbc1cbedd4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:45.345230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1082/2143] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:56:45 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:45.353797 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02619f1-dc49-4a2c-8760-0c6c57d740f0 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:45.363948 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02619f1-dc49-4a2c-8760-0c6c57d740f0 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:45.421309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378483f-793f-44f9-9339-05aee852b70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:45.421922 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378483f-793f-44f9-9339-05aee852b70a admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:45.422202 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378483f-793f-44f9-9339-05aee852b70a admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:45.422473 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378483f-793f-44f9-9339-05aee852b70a admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:45.423096 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8378483f-793f-44f9-9339-05aee852b70a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:45.425061 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8378483f-793f-44f9-9339-05aee852b70a admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:45.433614 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:45.448829 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1062/2144] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:56:45 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:56:45.452974 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555c83b-ade2-4358-b112-04cbc1cbedd4 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:45.463278 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555c83b-ade2-4358-b112-04cbc1cbedd4 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:45.468373 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b73301-8972-4130-aba8-3a5f8068986a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:45.468972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b73301-8972-4130-aba8-3a5f8068986a admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:45.469249 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b73301-8972-4130-aba8-3a5f8068986a admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:45.469519 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b73301-8972-4130-aba8-3a5f8068986a admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:45.470180 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b73301-8972-4130-aba8-3a5f8068986a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:45.471830 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b73301-8972-4130-aba8-3a5f8068986a admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:45.483160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1083/2145] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:56:45.487702 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378483f-793f-44f9-9339-05aee852b70a admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:45.500343 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378483f-793f-44f9-9339-05aee852b70a admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:45.505340 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1da9e-575b-479f-b3ff-7ea630393cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:45.505947 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1da9e-575b-479f-b3ff-7ea630393cce admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:45.506224 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1da9e-575b-479f-b3ff-7ea630393cce admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:45.506496 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1da9e-575b-479f-b3ff-7ea630393cce admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:45.507933 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1da9e-575b-479f-b3ff-7ea630393cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:45.509815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1da9e-575b-479f-b3ff-7ea630393cce admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:45.527543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1063/2146] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:56:45.528199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b73301-8972-4130-aba8-3a5f8068986a admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:45.539388 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b73301-8972-4130-aba8-3a5f8068986a admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:45.544155 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ac8a0-f492-483e-87fe-22f159ef7ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:45.544821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ac8a0-f492-483e-87fe-22f159ef7ebc admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:45.545257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ac8a0-f492-483e-87fe-22f159ef7ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:45.545641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ac8a0-f492-483e-87fe-22f159ef7ebc admin admin] PATH_INFO: `/v3/projects/bf20fa85589d48e98eb5162aa84378b2/users/63aca2239bc148b19d777ffef2350cf5/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:45.567980 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ac8a0-f492-483e-87fe-22f159ef7ebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf20fa85589d48e98eb5162aa84378b2, user_id=63aca2239bc148b19d777ffef2350cf5, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:45.572993 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ac8a0-f492-483e-87fe-22f159ef7ebc admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:45.589340 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1084/2147] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:45 2022] PUT /identity/v3/projects/bf20fa85589d48e98eb5162aa84378b2/users/63aca2239bc148b19d777ffef2350cf5/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:45.599504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1accf9-7d4d-4bef-b074-f3b59bb5e3df None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:45.599788 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1accf9-7d4d-4bef-b074-f3b59bb5e3df None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:45.600058 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1accf9-7d4d-4bef-b074-f3b59bb5e3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:45.630286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-5e1accf9-7d4d-4bef-b074-f3b59bb5e3df None None] MFA Rules not processed for user `63aca2239bc148b19d777ffef2350cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:45.631610 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-5e1accf9-7d4d-4bef-b074-f3b59bb5e3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:45.709527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1064/2148] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:45.714523 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ac8a0-f492-483e-87fe-22f159ef7ebc admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:45.723397 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:45.778290 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ac8a0-f492-483e-87fe-22f159ef7ebc admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:45.782923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c273c1e-f311-4cb0-a47b-e01d38099e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:45.783524 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c273c1e-f311-4cb0-a47b-e01d38099e28 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:45.783804 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c273c1e-f311-4cb0-a47b-e01d38099e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:45.784120 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c273c1e-f311-4cb0-a47b-e01d38099e28 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:45.784741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c273c1e-f311-4cb0-a47b-e01d38099e28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:45.786635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c273c1e-f311-4cb0-a47b-e01d38099e28 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:45.817838 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1085/2149] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:56:45 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:45.818965 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1accf9-7d4d-4bef-b074-f3b59bb5e3df None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:45.832239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1accf9-7d4d-4bef-b074-f3b59bb5e3df None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:45.924551 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066976f-514d-41f5-bb57-88f6612842dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:45.925261 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066976f-514d-41f5-bb57-88f6612842dc admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:45.925603 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066976f-514d-41f5-bb57-88f6612842dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:45.925925 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066976f-514d-41f5-bb57-88f6612842dc admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:45.926627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066976f-514d-41f5-bb57-88f6612842dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:45.929223 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066976f-514d-41f5-bb57-88f6612842dc admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:45.966549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1065/2150] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:56:45 2022] POST /identity/v3/users => generated 300 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:45.968439 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c273c1e-f311-4cb0-a47b-e01d38099e28 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:45.981907 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c273c1e-f311-4cb0-a47b-e01d38099e28 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:45.987233 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140a073-06a5-4ec0-86e2-295b25a7abb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:45.987899 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140a073-06a5-4ec0-86e2-295b25a7abb7 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:45.988864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140a073-06a5-4ec0-86e2-295b25a7abb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:45.988864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140a073-06a5-4ec0-86e2-295b25a7abb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:45.989505 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140a073-06a5-4ec0-86e2-295b25a7abb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:45.992763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140a073-06a5-4ec0-86e2-295b25a7abb7 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:46.007964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1086/2151] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:46.012060 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066976f-514d-41f5-bb57-88f6612842dc admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:46.023425 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066976f-514d-41f5-bb57-88f6612842dc admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:46.028889 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b253d65-f4c5-47fa-bb07-35d6ac5fedaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:46.029713 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b253d65-f4c5-47fa-bb07-35d6ac5fedaf admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:46.029996 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b253d65-f4c5-47fa-bb07-35d6ac5fedaf admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:46.030275 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b253d65-f4c5-47fa-bb07-35d6ac5fedaf admin admin] PATH_INFO: `/v3/projects/d09825ed91aa477eb0a189844f1cc3c9/users/0981f80eadd04cfb8cd90f4761da689e/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:46.048761 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b253d65-f4c5-47fa-bb07-35d6ac5fedaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d09825ed91aa477eb0a189844f1cc3c9, user_id=0981f80eadd04cfb8cd90f4761da689e, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:46.051549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b253d65-f4c5-47fa-bb07-35d6ac5fedaf admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:46.068809 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1066/2152] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:46 2022] PUT /identity/v3/projects/d09825ed91aa477eb0a189844f1cc3c9/users/0981f80eadd04cfb8cd90f4761da689e/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:46.072825 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140a073-06a5-4ec0-86e2-295b25a7abb7 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:46.082998 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140a073-06a5-4ec0-86e2-295b25a7abb7 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:46.142553 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41bd3c-88f2-4887-8848-bb029a9b1b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:46.143206 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41bd3c-88f2-4887-8848-bb029a9b1b86 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:46.143467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41bd3c-88f2-4887-8848-bb029a9b1b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:46.143750 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41bd3c-88f2-4887-8848-bb029a9b1b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:46.144412 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41bd3c-88f2-4887-8848-bb029a9b1b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:46.146009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41bd3c-88f2-4887-8848-bb029a9b1b86 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:46.160641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1087/2153] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:46.162094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b253d65-f4c5-47fa-bb07-35d6ac5fedaf admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:46.173030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b253d65-f4c5-47fa-bb07-35d6ac5fedaf admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:46.177898 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd00273-6142-4202-b387-2349862f6acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:46.178546 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd00273-6142-4202-b387-2349862f6acb admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:46.178849 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd00273-6142-4202-b387-2349862f6acb admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:46.179142 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd00273-6142-4202-b387-2349862f6acb admin admin] PATH_INFO: `/v3/projects/d09825ed91aa477eb0a189844f1cc3c9/users/0981f80eadd04cfb8cd90f4761da689e/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:46.181305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd00273-6142-4202-b387-2349862f6acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d09825ed91aa477eb0a189844f1cc3c9, user_id=0981f80eadd04cfb8cd90f4761da689e, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:46.183936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd00273-6142-4202-b387-2349862f6acb admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:46.194985 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1067/2154] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:46 2022] PUT /identity/v3/projects/d09825ed91aa477eb0a189844f1cc3c9/users/0981f80eadd04cfb8cd90f4761da689e/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:46.206120 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fc5ae-0cb5-4eb6-ba2d-10e627f2f4e4 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:46.206409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fc5ae-0cb5-4eb6-ba2d-10e627f2f4e4 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:46.206685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fc5ae-0cb5-4eb6-ba2d-10e627f2f4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:46.236802 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-559fc5ae-0cb5-4eb6-ba2d-10e627f2f4e4 None None] MFA Rules not processed for user `0981f80eadd04cfb8cd90f4761da689e`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:46.237943 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-559fc5ae-0cb5-4eb6-ba2d-10e627f2f4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:46.313995 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1088/2155] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:46 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:56:46.326032 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd00273-6142-4202-b387-2349862f6acb admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:46.327867 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-1bd00273-6142-4202-b387-2349862f6acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:46.354249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:46.396517 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd00273-6142-4202-b387-2349862f6acb admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:46.402318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92669bd2-4fda-444c-99bf-c09ea58ffcf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:46.402949 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92669bd2-4fda-444c-99bf-c09ea58ffcf8 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:46.403226 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92669bd2-4fda-444c-99bf-c09ea58ffcf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:46.403500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92669bd2-4fda-444c-99bf-c09ea58ffcf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:46.414699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92669bd2-4fda-444c-99bf-c09ea58ffcf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:46.416194 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92669bd2-4fda-444c-99bf-c09ea58ffcf8 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:46.426959 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1068/2156] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:56:46 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:56:46.458146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559fc5ae-0cb5-4eb6-ba2d-10e627f2f4e4 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:46.464153 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:46.544589 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559fc5ae-0cb5-4eb6-ba2d-10e627f2f4e4 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:46.550182 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f6c217-feb2-4b46-b5e1-3e84c0d470c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:46.551037 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6c217-feb2-4b46-b5e1-3e84c0d470c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:46.551315 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6c217-feb2-4b46-b5e1-3e84c0d470c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:46.551683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6c217-feb2-4b46-b5e1-3e84c0d470c3 admin admin] PATH_INFO: `/v3/users/63aca2239bc148b19d777ffef2350cf5` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:46.553313 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6c217-feb2-4b46-b5e1-3e84c0d470c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63aca2239bc148b19d777ffef2350cf5)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:46.555674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6c217-feb2-4b46-b5e1-3e84c0d470c3 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:46.619203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a6f6c217-feb2-4b46-b5e1-3e84c0d470c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63aca2239bc148b19d777ffef2350cf5'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:46.619793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a6f6c217-feb2-4b46-b5e1-3e84c0d470c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63aca2239bc148b19d777ffef2350cf5'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:46.633153 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a6f6c217-feb2-4b46-b5e1-3e84c0d470c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63aca2239bc148b19d777ffef2350cf5'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:46.640785 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a6f6c217-feb2-4b46-b5e1-3e84c0d470c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63aca2239bc148b19d777ffef2350cf5'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:46.652780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a6f6c217-feb2-4b46-b5e1-3e84c0d470c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63aca2239bc148b19d777ffef2350cf5'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:46.659400 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1089/2157] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:56:46 2022] DELETE /identity/v3/users/63aca2239bc148b19d777ffef2350cf5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:46.666034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76cba49-926c-452d-b29d-3739a6746d81 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:46.666034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76cba49-926c-452d-b29d-3739a6746d81 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:46.666034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76cba49-926c-452d-b29d-3739a6746d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:46.687715 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-f76cba49-926c-452d-b29d-3739a6746d81 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:46.689386 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-f76cba49-926c-452d-b29d-3739a6746d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:46.751880 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1069/2158] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:46.761315 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f6c217-feb2-4b46-b5e1-3e84c0d470c3 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:46.762731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a6f6c217-feb2-4b46-b5e1-3e84c0d470c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:46.822023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f6c217-feb2-4b46-b5e1-3e84c0d470c3 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:46.825780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737598b-2802-48be-beec-192f72da6d26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:46.826399 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737598b-2802-48be-beec-192f72da6d26 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:46.826680 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737598b-2802-48be-beec-192f72da6d26 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:46.827013 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737598b-2802-48be-beec-192f72da6d26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:46.838521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737598b-2802-48be-beec-192f72da6d26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:46.840148 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737598b-2802-48be-beec-192f72da6d26 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:46.850331 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1090/2159] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:56:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:56:47.273092 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76cba49-926c-452d-b29d-3739a6746d81 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:47.275260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-f76cba49-926c-452d-b29d-3739a6746d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:47.290194 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76cba49-926c-452d-b29d-3739a6746d81 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:47.295342 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731c56f-b0ed-4ac4-bbfe-4eea5ba9e1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:47.296320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731c56f-b0ed-4ac4-bbfe-4eea5ba9e1a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:47.296793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731c56f-b0ed-4ac4-bbfe-4eea5ba9e1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:47.297079 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731c56f-b0ed-4ac4-bbfe-4eea5ba9e1a4 admin admin] PATH_INFO: `/v3/projects/bf20fa85589d48e98eb5162aa84378b2` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:47.298533 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731c56f-b0ed-4ac4-bbfe-4eea5ba9e1a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf20fa85589d48e98eb5162aa84378b2)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:47.300813 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731c56f-b0ed-4ac4-bbfe-4eea5ba9e1a4 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:47.365895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-9731c56f-b0ed-4ac4-bbfe-4eea5ba9e1a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf20fa85589d48e98eb5162aa84378b2', 'request_id': 'req-9731c56f-b0ed-4ac4-bbfe-4eea5ba9e1a4'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:47.374585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-9731c56f-b0ed-4ac4-bbfe-4eea5ba9e1a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf20fa85589d48e98eb5162aa84378b2', 'request_id': 'req-9731c56f-b0ed-4ac4-bbfe-4eea5ba9e1a4'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:47.381077 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-9731c56f-b0ed-4ac4-bbfe-4eea5ba9e1a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf20fa85589d48e98eb5162aa84378b2', 'request_id': 'req-9731c56f-b0ed-4ac4-bbfe-4eea5ba9e1a4'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:47.392772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-9731c56f-b0ed-4ac4-bbfe-4eea5ba9e1a4 admin admin] The token cache is being invalidate because project bf20fa85589d48e98eb5162aa84378b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:56:47.393059 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-9731c56f-b0ed-4ac4-bbfe-4eea5ba9e1a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:47.393810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-9731c56f-b0ed-4ac4-bbfe-4eea5ba9e1a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:47.395304 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1070/2160] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:56:47 2022] DELETE /identity/v3/projects/bf20fa85589d48e98eb5162aa84378b2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:47.400399 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737598b-2802-48be-beec-192f72da6d26 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:47.410354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737598b-2802-48be-beec-192f72da6d26 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:47.433356 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:47.474258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53efaecd-a32d-4c2b-9e40-26a051275aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:47.474969 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efaecd-a32d-4c2b-9e40-26a051275aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:47.475252 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efaecd-a32d-4c2b-9e40-26a051275aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:47.475547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efaecd-a32d-4c2b-9e40-26a051275aa9 admin admin] PATH_INFO: `/v3/users/0981f80eadd04cfb8cd90f4761da689e` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:47.476765 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53efaecd-a32d-4c2b-9e40-26a051275aa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0981f80eadd04cfb8cd90f4761da689e)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:47.478846 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53efaecd-a32d-4c2b-9e40-26a051275aa9 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:47.527294 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-53efaecd-a32d-4c2b-9e40-26a051275aa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0981f80eadd04cfb8cd90f4761da689e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:47.527946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-53efaecd-a32d-4c2b-9e40-26a051275aa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0981f80eadd04cfb8cd90f4761da689e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:47.539911 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-53efaecd-a32d-4c2b-9e40-26a051275aa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0981f80eadd04cfb8cd90f4761da689e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:47.546797 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-53efaecd-a32d-4c2b-9e40-26a051275aa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0981f80eadd04cfb8cd90f4761da689e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:47.555227 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-53efaecd-a32d-4c2b-9e40-26a051275aa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0981f80eadd04cfb8cd90f4761da689e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:47.561413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1091/2161] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:56:47 2022] DELETE /identity/v3/users/0981f80eadd04cfb8cd90f4761da689e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:47.937291 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731c56f-b0ed-4ac4-bbfe-4eea5ba9e1a4 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:47.938704 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-9731c56f-b0ed-4ac4-bbfe-4eea5ba9e1a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:47.955748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731c56f-b0ed-4ac4-bbfe-4eea5ba9e1a4 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:48.001816 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:48.015945 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844b0d5-995d-4843-a742-f4e272927816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:48.016608 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844b0d5-995d-4843-a742-f4e272927816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:48.016887 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844b0d5-995d-4843-a742-f4e272927816 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:48.017160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844b0d5-995d-4843-a742-f4e272927816 admin admin] PATH_INFO: `/v3/projects/d09825ed91aa477eb0a189844f1cc3c9` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:48.018278 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1844b0d5-995d-4843-a742-f4e272927816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d09825ed91aa477eb0a189844f1cc3c9)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:48.020222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1844b0d5-995d-4843-a742-f4e272927816 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:48.069411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1844b0d5-995d-4843-a742-f4e272927816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd09825ed91aa477eb0a189844f1cc3c9', 'request_id': 'req-1844b0d5-995d-4843-a742-f4e272927816'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:48.077378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1844b0d5-995d-4843-a742-f4e272927816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd09825ed91aa477eb0a189844f1cc3c9', 'request_id': 'req-1844b0d5-995d-4843-a742-f4e272927816'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:48.084233 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1844b0d5-995d-4843-a742-f4e272927816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd09825ed91aa477eb0a189844f1cc3c9', 'request_id': 'req-1844b0d5-995d-4843-a742-f4e272927816'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:48.095212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1844b0d5-995d-4843-a742-f4e272927816 admin admin] The token cache is being invalidate because project d09825ed91aa477eb0a189844f1cc3c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:56:48.095500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1844b0d5-995d-4843-a742-f4e272927816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:48.096172 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1844b0d5-995d-4843-a742-f4e272927816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:48.097638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1071/2162] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:56:47 2022] DELETE /identity/v3/projects/d09825ed91aa477eb0a189844f1cc3c9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:48.103241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027159d-c7cd-4f41-806d-561cd1d500f3 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:48.103532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027159d-c7cd-4f41-806d-561cd1d500f3 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:48.103811 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027159d-c7cd-4f41-806d-561cd1d500f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:48.124334 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-d027159d-c7cd-4f41-806d-561cd1d500f3 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:48.125534 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d027159d-c7cd-4f41-806d-561cd1d500f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:48.186420 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1092/2163] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:48.194167 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5842e5-0de2-469a-8326-e1ecec02def5 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:48.194697 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5842e5-0de2-469a-8326-e1ecec02def5 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:48.195319 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5842e5-0de2-469a-8326-e1ecec02def5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:48.215540 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-df5842e5-0de2-469a-8326-e1ecec02def5 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:48.216793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-df5842e5-0de2-469a-8326-e1ecec02def5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:48.226201 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1072/2164] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:48.230612 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027159d-c7cd-4f41-806d-561cd1d500f3 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:48.240278 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027159d-c7cd-4f41-806d-561cd1d500f3 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:48.246281 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf734ca7-987f-47e3-a0ad-e058e9dbfccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:48.246886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf734ca7-987f-47e3-a0ad-e058e9dbfccc admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:48.247163 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf734ca7-987f-47e3-a0ad-e058e9dbfccc admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:48.247431 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf734ca7-987f-47e3-a0ad-e058e9dbfccc admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:48.248075 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf734ca7-987f-47e3-a0ad-e058e9dbfccc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:56:48.248399 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf734ca7-987f-47e3-a0ad-e058e9dbfccc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:48.250004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf734ca7-987f-47e3-a0ad-e058e9dbfccc admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:48.261758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1093/2165] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:56:48.266174 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5842e5-0de2-469a-8326-e1ecec02def5 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:48.275936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5842e5-0de2-469a-8326-e1ecec02def5 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:48.280688 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a63f24-d884-4cdb-9a31-10e70ae69ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:48.281299 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a63f24-d884-4cdb-9a31-10e70ae69ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:48.281582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a63f24-d884-4cdb-9a31-10e70ae69ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:48.281856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a63f24-d884-4cdb-9a31-10e70ae69ee6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:48.282492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a63f24-d884-4cdb-9a31-10e70ae69ee6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:48.284439 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a63f24-d884-4cdb-9a31-10e70ae69ee6 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:48.318345 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1073/2166] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:56:48 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:48.323310 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf734ca7-987f-47e3-a0ad-e058e9dbfccc admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:48.333464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf734ca7-987f-47e3-a0ad-e058e9dbfccc admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:48.362347 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:48.391929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aad013-617c-4b32-ac09-aa177082e4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:48.392575 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aad013-617c-4b32-ac09-aa177082e4fe admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:48.392875 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aad013-617c-4b32-ac09-aa177082e4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:48.393134 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aad013-617c-4b32-ac09-aa177082e4fe admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:48.393772 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aad013-617c-4b32-ac09-aa177082e4fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:48.395688 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aad013-617c-4b32-ac09-aa177082e4fe admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:48.418359 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1094/2167] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:56:48 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:48.422462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a63f24-d884-4cdb-9a31-10e70ae69ee6 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:48.433342 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a63f24-d884-4cdb-9a31-10e70ae69ee6 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:48.437856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9fc674-5a06-466c-9a23-cb68d86d036c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:48.438446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9fc674-5a06-466c-9a23-cb68d86d036c admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:48.438722 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9fc674-5a06-466c-9a23-cb68d86d036c admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:48.438993 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9fc674-5a06-466c-9a23-cb68d86d036c admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:48.439620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9fc674-5a06-466c-9a23-cb68d86d036c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:48.441246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9fc674-5a06-466c-9a23-cb68d86d036c admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:48.451863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1074/2168] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:48.455630 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aad013-617c-4b32-ac09-aa177082e4fe admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:48.466334 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aad013-617c-4b32-ac09-aa177082e4fe admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:48.470916 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97a795-45b9-4468-81b6-f6afcf8b9d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:48.471509 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97a795-45b9-4468-81b6-f6afcf8b9d27 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:48.471790 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97a795-45b9-4468-81b6-f6afcf8b9d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:48.472063 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97a795-45b9-4468-81b6-f6afcf8b9d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:48.472721 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97a795-45b9-4468-81b6-f6afcf8b9d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:48.474301 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97a795-45b9-4468-81b6-f6afcf8b9d27 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:48.485011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1095/2169] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:48.489200 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9fc674-5a06-466c-9a23-cb68d86d036c admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:48.499287 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9fc674-5a06-466c-9a23-cb68d86d036c admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:48.504039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3a2dd-295e-4674-af3b-d4d96d5e0e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:48.504714 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3a2dd-295e-4674-af3b-d4d96d5e0e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:48.505007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3a2dd-295e-4674-af3b-d4d96d5e0e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:48.505284 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3a2dd-295e-4674-af3b-d4d96d5e0e51 admin admin] PATH_INFO: `/v3/projects/1a41d0a52e054bb58c6982d6c40dedae/users/533d8c2f59b04c04b1d894c3026cef4b/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:48.520867 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3a2dd-295e-4674-af3b-d4d96d5e0e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a41d0a52e054bb58c6982d6c40dedae, user_id=533d8c2f59b04c04b1d894c3026cef4b, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:48.524710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3a2dd-295e-4674-af3b-d4d96d5e0e51 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:48.537290 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1075/2170] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:48 2022] PUT /identity/v3/projects/1a41d0a52e054bb58c6982d6c40dedae/users/533d8c2f59b04c04b1d894c3026cef4b/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:48.547511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b953b3e-f659-4bb4-a003-84dc07c2903a None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:48.547796 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b953b3e-f659-4bb4-a003-84dc07c2903a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:48.548070 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b953b3e-f659-4bb4-a003-84dc07c2903a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:48.575799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-4b953b3e-f659-4bb4-a003-84dc07c2903a None None] MFA Rules not processed for user `533d8c2f59b04c04b1d894c3026cef4b`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:48.576989 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4b953b3e-f659-4bb4-a003-84dc07c2903a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:48.640239 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1096/2171] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:48.644708 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3a2dd-295e-4674-af3b-d4d96d5e0e51 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:48.706383 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3a2dd-295e-4674-af3b-d4d96d5e0e51 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:48.711952 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7739-11b3-4eab-81ba-c72d4c83495b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:48.712588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7739-11b3-4eab-81ba-c72d4c83495b admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:48.712872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7739-11b3-4eab-81ba-c72d4c83495b admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:48.713148 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7739-11b3-4eab-81ba-c72d4c83495b admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:48.713788 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e7739-11b3-4eab-81ba-c72d4c83495b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:48.715697 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e7739-11b3-4eab-81ba-c72d4c83495b admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:48.735856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:48.745056 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1076/2172] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:56:48 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:48.748902 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b953b3e-f659-4bb4-a003-84dc07c2903a None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:48.760396 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b953b3e-f659-4bb4-a003-84dc07c2903a None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:48.816707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ae501-3432-44f3-a8e0-54a9fbc44102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:48.817311 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ae501-3432-44f3-a8e0-54a9fbc44102 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:48.817586 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ae501-3432-44f3-a8e0-54a9fbc44102 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:48.817854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ae501-3432-44f3-a8e0-54a9fbc44102 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:48.818472 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ae501-3432-44f3-a8e0-54a9fbc44102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:48.821276 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ae501-3432-44f3-a8e0-54a9fbc44102 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:48.842374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1097/2173] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:56:48 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:56:48.846266 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7739-11b3-4eab-81ba-c72d4c83495b admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:48.856490 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7739-11b3-4eab-81ba-c72d4c83495b admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:48.861055 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a60cbc-2ed7-49c2-8fd7-834af67a902c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:48.861643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a60cbc-2ed7-49c2-8fd7-834af67a902c admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:48.861922 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a60cbc-2ed7-49c2-8fd7-834af67a902c admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:48.862188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a60cbc-2ed7-49c2-8fd7-834af67a902c admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:48.862810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a60cbc-2ed7-49c2-8fd7-834af67a902c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:48.864437 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a60cbc-2ed7-49c2-8fd7-834af67a902c admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:48.875843 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1077/2174] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:48.880037 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ae501-3432-44f3-a8e0-54a9fbc44102 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:48.891438 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ae501-3432-44f3-a8e0-54a9fbc44102 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:48.896229 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe2232-ac19-4482-b2bb-9ed8565b402e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:48.896889 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe2232-ac19-4482-b2bb-9ed8565b402e admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:48.897169 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe2232-ac19-4482-b2bb-9ed8565b402e admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:48.897496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe2232-ac19-4482-b2bb-9ed8565b402e admin admin] PATH_INFO: `/v3/projects/e9b8aa6c0e1342f9ba5bb87f9f9092f2/users/1bcd05d2ab2a41f9956ec59cb79381f2/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:48.912378 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe2232-ac19-4482-b2bb-9ed8565b402e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9b8aa6c0e1342f9ba5bb87f9f9092f2, user_id=1bcd05d2ab2a41f9956ec59cb79381f2, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:48.915080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe2232-ac19-4482-b2bb-9ed8565b402e admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:48.928223 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1098/2175] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:48 2022] PUT /identity/v3/projects/e9b8aa6c0e1342f9ba5bb87f9f9092f2/users/1bcd05d2ab2a41f9956ec59cb79381f2/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:48.932912 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a60cbc-2ed7-49c2-8fd7-834af67a902c admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:48.942833 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a60cbc-2ed7-49c2-8fd7-834af67a902c admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:48.999188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1dd8e-c82f-485a-8601-6847cd5de89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:48.999788 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1dd8e-c82f-485a-8601-6847cd5de89e admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:49.000069 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1dd8e-c82f-485a-8601-6847cd5de89e admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:49.000378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1dd8e-c82f-485a-8601-6847cd5de89e admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:49.001009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e1dd8e-c82f-485a-8601-6847cd5de89e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:49.002603 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e1dd8e-c82f-485a-8601-6847cd5de89e admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:49.013079 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1078/2176] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:49.017207 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe2232-ac19-4482-b2bb-9ed8565b402e admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:49.023797 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:49.027815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe2232-ac19-4482-b2bb-9ed8565b402e admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:49.033674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71257d-ab82-4c84-80ec-07cbd139b514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:49.034357 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71257d-ab82-4c84-80ec-07cbd139b514 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:49.034641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71257d-ab82-4c84-80ec-07cbd139b514 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:49.034918 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71257d-ab82-4c84-80ec-07cbd139b514 admin admin] PATH_INFO: `/v3/projects/e9b8aa6c0e1342f9ba5bb87f9f9092f2/users/1bcd05d2ab2a41f9956ec59cb79381f2/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:49.036895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71257d-ab82-4c84-80ec-07cbd139b514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9b8aa6c0e1342f9ba5bb87f9f9092f2, user_id=1bcd05d2ab2a41f9956ec59cb79381f2, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:49.039723 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d71257d-ab82-4c84-80ec-07cbd139b514 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:49.050176 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1099/2177] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:49 2022] PUT /identity/v3/projects/e9b8aa6c0e1342f9ba5bb87f9f9092f2/users/1bcd05d2ab2a41f9956ec59cb79381f2/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:49.115291 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0e63f-316b-4615-b578-56f2ef72483d None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:49.115512 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0e63f-316b-4615-b578-56f2ef72483d None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:49.115764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0e63f-316b-4615-b578-56f2ef72483d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:49.144456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-33f0e63f-316b-4615-b578-56f2ef72483d None None] MFA Rules not processed for user `1bcd05d2ab2a41f9956ec59cb79381f2`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:49.145697 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-33f0e63f-316b-4615-b578-56f2ef72483d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:49.213765 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1079/2178] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:49.224912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71257d-ab82-4c84-80ec-07cbd139b514 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:49.227619 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-2d71257d-ab82-4c84-80ec-07cbd139b514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:49.287029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71257d-ab82-4c84-80ec-07cbd139b514 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:49.291434 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a08f8d-cd85-40cf-abf8-cb6376956c38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:49.292041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a08f8d-cd85-40cf-abf8-cb6376956c38 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:49.292380 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a08f8d-cd85-40cf-abf8-cb6376956c38 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:49.292655 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a08f8d-cd85-40cf-abf8-cb6376956c38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:49.301652 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a08f8d-cd85-40cf-abf8-cb6376956c38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:49.303065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a08f8d-cd85-40cf-abf8-cb6376956c38 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:49.313149 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1100/2179] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:56:49 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:56:49.420728 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0e63f-316b-4615-b578-56f2ef72483d None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:49.430963 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0e63f-316b-4615-b578-56f2ef72483d None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:49.434891 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d8e02-b528-43ac-b986-a51df7807dc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:49.435577 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d8e02-b528-43ac-b986-a51df7807dc3 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:49.435925 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d8e02-b528-43ac-b986-a51df7807dc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:49.436226 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d8e02-b528-43ac-b986-a51df7807dc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:49.498506 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d8e02-b528-43ac-b986-a51df7807dc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:49.500608 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d8e02-b528-43ac-b986-a51df7807dc3 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:49.510549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1080/2180] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:56:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:56:52.100526 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a08f8d-cd85-40cf-abf8-cb6376956c38 service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:52.172033 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a08f8d-cd85-40cf-abf8-cb6376956c38 service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:52.176597 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee3346-4c81-49bb-9f24-9326f306d13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:52.177225 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee3346-4c81-49bb-9f24-9326f306d13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:52.177499 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee3346-4c81-49bb-9f24-9326f306d13b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:52.177769 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee3346-4c81-49bb-9f24-9326f306d13b admin admin] PATH_INFO: `/v3/users/533d8c2f59b04c04b1d894c3026cef4b` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:52.179399 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee3346-4c81-49bb-9f24-9326f306d13b admin admin] RBAC: Authorizing `identity:delete_user(user_id=533d8c2f59b04c04b1d894c3026cef4b)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:52.181381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee3346-4c81-49bb-9f24-9326f306d13b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:52.230639 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-19ee3346-4c81-49bb-9f24-9326f306d13b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '533d8c2f59b04c04b1d894c3026cef4b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:52.231298 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-19ee3346-4c81-49bb-9f24-9326f306d13b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '533d8c2f59b04c04b1d894c3026cef4b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:52.244457 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-19ee3346-4c81-49bb-9f24-9326f306d13b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '533d8c2f59b04c04b1d894c3026cef4b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:52.251339 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-19ee3346-4c81-49bb-9f24-9326f306d13b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '533d8c2f59b04c04b1d894c3026cef4b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:52.259043 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-19ee3346-4c81-49bb-9f24-9326f306d13b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '533d8c2f59b04c04b1d894c3026cef4b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:52.265168 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1101/2181] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:56:52 2022] DELETE /identity/v3/users/533d8c2f59b04c04b1d894c3026cef4b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:52.269687 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2338f13-17bf-4f26-8086-f5ba1778571a None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:52.270187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2338f13-17bf-4f26-8086-f5ba1778571a None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:52.270283 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2338f13-17bf-4f26-8086-f5ba1778571a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:52.283489 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=100 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:52.290202 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-c2338f13-17bf-4f26-8086-f5ba1778571a None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:52.291406 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c2338f13-17bf-4f26-8086-f5ba1778571a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:52.349893 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1081/2182] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:52.360252 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee3346-4c81-49bb-9f24-9326f306d13b admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:52.361677 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-19ee3346-4c81-49bb-9f24-9326f306d13b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:52.376594 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:52.422900 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee3346-4c81-49bb-9f24-9326f306d13b admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:52.426551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d7ebf-ce1f-4e7e-90b0-c5e5230cdbcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:52.427157 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d7ebf-ce1f-4e7e-90b0-c5e5230cdbcb service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:52.427435 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d7ebf-ce1f-4e7e-90b0-c5e5230cdbcb service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:52.427706 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d7ebf-ce1f-4e7e-90b0-c5e5230cdbcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:52.437720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d7ebf-ce1f-4e7e-90b0-c5e5230cdbcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:52.439145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d7ebf-ce1f-4e7e-90b0-c5e5230cdbcb service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:52.449225 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1102/2183] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:56:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:56:52.727305 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2338f13-17bf-4f26-8086-f5ba1778571a None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:52.728848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c2338f13-17bf-4f26-8086-f5ba1778571a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:52.743331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2338f13-17bf-4f26-8086-f5ba1778571a None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:52.748519 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535dc50-5b5e-4d8b-9fa2-0f294a0d3183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:52.749172 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535dc50-5b5e-4d8b-9fa2-0f294a0d3183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:52.749455 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535dc50-5b5e-4d8b-9fa2-0f294a0d3183 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:52.750320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535dc50-5b5e-4d8b-9fa2-0f294a0d3183 admin admin] PATH_INFO: `/v3/projects/1a41d0a52e054bb58c6982d6c40dedae` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:52.751687 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535dc50-5b5e-4d8b-9fa2-0f294a0d3183 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a41d0a52e054bb58c6982d6c40dedae)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:52.753664 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535dc50-5b5e-4d8b-9fa2-0f294a0d3183 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:52.801443 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-d535dc50-5b5e-4d8b-9fa2-0f294a0d3183 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a41d0a52e054bb58c6982d6c40dedae', 'request_id': 'req-d535dc50-5b5e-4d8b-9fa2-0f294a0d3183'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:52.809220 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-d535dc50-5b5e-4d8b-9fa2-0f294a0d3183 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a41d0a52e054bb58c6982d6c40dedae', 'request_id': 'req-d535dc50-5b5e-4d8b-9fa2-0f294a0d3183'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:52.815481 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-d535dc50-5b5e-4d8b-9fa2-0f294a0d3183 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a41d0a52e054bb58c6982d6c40dedae', 'request_id': 'req-d535dc50-5b5e-4d8b-9fa2-0f294a0d3183'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:52.826757 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-d535dc50-5b5e-4d8b-9fa2-0f294a0d3183 admin admin] The token cache is being invalidate because project 1a41d0a52e054bb58c6982d6c40dedae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:56:52.827042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-d535dc50-5b5e-4d8b-9fa2-0f294a0d3183 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:52.827661 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-d535dc50-5b5e-4d8b-9fa2-0f294a0d3183 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:52.829149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1082/2184] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:56:52 2022] DELETE /identity/v3/projects/1a41d0a52e054bb58c6982d6c40dedae => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:52.833625 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d7ebf-ce1f-4e7e-90b0-c5e5230cdbcb service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:52.843618 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d7ebf-ce1f-4e7e-90b0-c5e5230cdbcb service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:52.904763 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04c2ef-1140-4be6-90fb-04b12dd219e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:52.905397 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04c2ef-1140-4be6-90fb-04b12dd219e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:52.905678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04c2ef-1140-4be6-90fb-04b12dd219e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:52.905946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04c2ef-1140-4be6-90fb-04b12dd219e9 admin admin] PATH_INFO: `/v3/users/1bcd05d2ab2a41f9956ec59cb79381f2` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:52.907071 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04c2ef-1140-4be6-90fb-04b12dd219e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bcd05d2ab2a41f9956ec59cb79381f2)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:52.909020 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04c2ef-1140-4be6-90fb-04b12dd219e9 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:52.956407 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-df04c2ef-1140-4be6-90fb-04b12dd219e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bcd05d2ab2a41f9956ec59cb79381f2'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:52.957062 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-df04c2ef-1140-4be6-90fb-04b12dd219e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bcd05d2ab2a41f9956ec59cb79381f2'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:52.970027 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-df04c2ef-1140-4be6-90fb-04b12dd219e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bcd05d2ab2a41f9956ec59cb79381f2'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:52.977792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-df04c2ef-1140-4be6-90fb-04b12dd219e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bcd05d2ab2a41f9956ec59cb79381f2'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:52.985738 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-df04c2ef-1140-4be6-90fb-04b12dd219e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bcd05d2ab2a41f9956ec59cb79381f2'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:52.992949 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1103/2185] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:56:52 2022] DELETE /identity/v3/users/1bcd05d2ab2a41f9956ec59cb79381f2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:53.264319 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535dc50-5b5e-4d8b-9fa2-0f294a0d3183 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:53.265770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d535dc50-5b5e-4d8b-9fa2-0f294a0d3183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:53.280317 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535dc50-5b5e-4d8b-9fa2-0f294a0d3183 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:53.339836 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527715d-7c73-461b-b2ca-cd134c18717b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:53.340530 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527715d-7c73-461b-b2ca-cd134c18717b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:53.340944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527715d-7c73-461b-b2ca-cd134c18717b admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:53.341224 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527715d-7c73-461b-b2ca-cd134c18717b admin admin] PATH_INFO: `/v3/projects/e9b8aa6c0e1342f9ba5bb87f9f9092f2` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:53.342353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527715d-7c73-461b-b2ca-cd134c18717b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9b8aa6c0e1342f9ba5bb87f9f9092f2)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:53.344285 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527715d-7c73-461b-b2ca-cd134c18717b admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:53.379707 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:53.393275 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8527715d-7c73-461b-b2ca-cd134c18717b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9b8aa6c0e1342f9ba5bb87f9f9092f2', 'request_id': 'req-8527715d-7c73-461b-b2ca-cd134c18717b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:53.401132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8527715d-7c73-461b-b2ca-cd134c18717b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9b8aa6c0e1342f9ba5bb87f9f9092f2', 'request_id': 'req-8527715d-7c73-461b-b2ca-cd134c18717b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:53.407398 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8527715d-7c73-461b-b2ca-cd134c18717b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9b8aa6c0e1342f9ba5bb87f9f9092f2', 'request_id': 'req-8527715d-7c73-461b-b2ca-cd134c18717b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:53.418426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8527715d-7c73-461b-b2ca-cd134c18717b admin admin] The token cache is being invalidate because project e9b8aa6c0e1342f9ba5bb87f9f9092f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:56:53.418719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8527715d-7c73-461b-b2ca-cd134c18717b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:53.419358 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-8527715d-7c73-461b-b2ca-cd134c18717b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:53.420879 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1083/2186] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:56:53 2022] DELETE /identity/v3/projects/e9b8aa6c0e1342f9ba5bb87f9f9092f2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:53.425748 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97dce33-4ba5-42b1-a8e9-a226b17af419 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:53.426217 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97dce33-4ba5-42b1-a8e9-a226b17af419 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:53.426329 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97dce33-4ba5-42b1-a8e9-a226b17af419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:53.445884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-f97dce33-4ba5-42b1-a8e9-a226b17af419 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:53.447089 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f97dce33-4ba5-42b1-a8e9-a226b17af419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:53.477761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:53.508411 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1104/2187] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:53.515972 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb450ca-631d-4d42-9d76-293915112cf4 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:53.516279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb450ca-631d-4d42-9d76-293915112cf4 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:53.516551 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb450ca-631d-4d42-9d76-293915112cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:53.536888 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-fdb450ca-631d-4d42-9d76-293915112cf4 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:53.538091 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-fdb450ca-631d-4d42-9d76-293915112cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:53.546742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1084/2188] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:53.550867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97dce33-4ba5-42b1-a8e9-a226b17af419 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:53.560979 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97dce33-4ba5-42b1-a8e9-a226b17af419 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:53.565579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f50fcb-cd78-4c90-8ec7-aa187073d76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:53.566177 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f50fcb-cd78-4c90-8ec7-aa187073d76e admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:53.566458 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f50fcb-cd78-4c90-8ec7-aa187073d76e admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:53.566732 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f50fcb-cd78-4c90-8ec7-aa187073d76e admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:53.567377 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f50fcb-cd78-4c90-8ec7-aa187073d76e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:56:53.567682 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f50fcb-cd78-4c90-8ec7-aa187073d76e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:53.569312 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f50fcb-cd78-4c90-8ec7-aa187073d76e admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:53.581214 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1105/2189] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:56:53.585738 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb450ca-631d-4d42-9d76-293915112cf4 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:53.594923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb450ca-631d-4d42-9d76-293915112cf4 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:53.599539 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec87d4-cb51-4524-834c-2686cc8c2198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:53.600165 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec87d4-cb51-4524-834c-2686cc8c2198 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:53.600449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec87d4-cb51-4524-834c-2686cc8c2198 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:53.600745 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec87d4-cb51-4524-834c-2686cc8c2198 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:53.601364 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec87d4-cb51-4524-834c-2686cc8c2198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:53.603261 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec87d4-cb51-4524-834c-2686cc8c2198 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:53.631359 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1085/2190] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:56:53 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:53.636670 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f50fcb-cd78-4c90-8ec7-aa187073d76e admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:53.646784 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f50fcb-cd78-4c90-8ec7-aa187073d76e admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:53.703189 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf69005-3597-411c-badb-8732889114c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:53.703790 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf69005-3597-411c-badb-8732889114c5 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:53.704072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf69005-3597-411c-badb-8732889114c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:53.704374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf69005-3597-411c-badb-8732889114c5 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:53.704998 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf69005-3597-411c-badb-8732889114c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:53.706920 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf69005-3597-411c-badb-8732889114c5 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:53.729691 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1106/2191] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:56:53 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:56:53.733926 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec87d4-cb51-4524-834c-2686cc8c2198 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:53.745128 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec87d4-cb51-4524-834c-2686cc8c2198 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:53.749737 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5acde-297d-4501-a5e7-8b01b9fcd7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:53.750340 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5acde-297d-4501-a5e7-8b01b9fcd7ac admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:53.750620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5acde-297d-4501-a5e7-8b01b9fcd7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:53.750891 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5acde-297d-4501-a5e7-8b01b9fcd7ac admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:53.751544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5acde-297d-4501-a5e7-8b01b9fcd7ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:53.753192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5acde-297d-4501-a5e7-8b01b9fcd7ac admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:53.763292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1086/2192] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:53.767238 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf69005-3597-411c-badb-8732889114c5 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:53.777414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf69005-3597-411c-badb-8732889114c5 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:53.782116 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad9f4d-34c4-4374-bac4-1cb21818d38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:53.782647 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad9f4d-34c4-4374-bac4-1cb21818d38f admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:53.782922 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad9f4d-34c4-4374-bac4-1cb21818d38f admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:53.783195 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad9f4d-34c4-4374-bac4-1cb21818d38f admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:53.783869 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad9f4d-34c4-4374-bac4-1cb21818d38f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:53.785462 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad9f4d-34c4-4374-bac4-1cb21818d38f admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:53.796636 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1107/2193] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:53.801035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5acde-297d-4501-a5e7-8b01b9fcd7ac admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:53.812150 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5acde-297d-4501-a5e7-8b01b9fcd7ac admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:53.816675 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19b37a-c384-48ba-ba4d-8ada7a6b0120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:53.817319 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19b37a-c384-48ba-ba4d-8ada7a6b0120 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:53.817598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19b37a-c384-48ba-ba4d-8ada7a6b0120 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:53.817870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19b37a-c384-48ba-ba4d-8ada7a6b0120 admin admin] PATH_INFO: `/v3/projects/f77a2a1f5bd4485f8f97470774c47713/users/e2aaecabea044b1289c722070e77d520/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:53.832050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f19b37a-c384-48ba-ba4d-8ada7a6b0120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f77a2a1f5bd4485f8f97470774c47713, user_id=e2aaecabea044b1289c722070e77d520, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:53.834963 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f19b37a-c384-48ba-ba4d-8ada7a6b0120 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:53.847114 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1087/2194] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:53 2022] PUT /identity/v3/projects/f77a2a1f5bd4485f8f97470774c47713/users/e2aaecabea044b1289c722070e77d520/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:53.857548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b9661-82ab-4306-af3b-3a7036cca7de None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:53.857838 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b9661-82ab-4306-af3b-3a7036cca7de None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:53.858426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b9661-82ab-4306-af3b-3a7036cca7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:53.889177 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-e61b9661-82ab-4306-af3b-3a7036cca7de None None] MFA Rules not processed for user `e2aaecabea044b1289c722070e77d520`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:53.890512 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e61b9661-82ab-4306-af3b-3a7036cca7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:53.956418 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1108/2195] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:53.961191 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19b37a-c384-48ba-ba4d-8ada7a6b0120 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:54.034663 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19b37a-c384-48ba-ba4d-8ada7a6b0120 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:54.039870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec0442-f667-46f5-8967-dc9fa51e5ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:54.040502 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec0442-f667-46f5-8967-dc9fa51e5ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:54.040781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec0442-f667-46f5-8967-dc9fa51e5ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:54.041055 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec0442-f667-46f5-8967-dc9fa51e5ca9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:54.041693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec0442-f667-46f5-8967-dc9fa51e5ca9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:54.043633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec0442-f667-46f5-8967-dc9fa51e5ca9 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:54.077940 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1088/2196] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:56:53 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:56:54.081800 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b9661-82ab-4306-af3b-3a7036cca7de None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:54.094641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b9661-82ab-4306-af3b-3a7036cca7de None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:54.130137 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:54.172929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003f75e-59ab-4c24-8032-8e72466a91cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:54.173562 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003f75e-59ab-4c24-8032-8e72466a91cd admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:54.173843 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003f75e-59ab-4c24-8032-8e72466a91cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:54.174115 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003f75e-59ab-4c24-8032-8e72466a91cd admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:54.174952 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1003f75e-59ab-4c24-8032-8e72466a91cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:54.176969 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1003f75e-59ab-4c24-8032-8e72466a91cd admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:54.208714 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1109/2197] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:56:54 2022] POST /identity/v3/users => generated 294 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:54.213005 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec0442-f667-46f5-8967-dc9fa51e5ca9 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:54.231658 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec0442-f667-46f5-8967-dc9fa51e5ca9 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:54.237577 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b9d8e-720b-4c17-a058-40c367558712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:54.238198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b9d8e-720b-4c17-a058-40c367558712 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:54.238486 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b9d8e-720b-4c17-a058-40c367558712 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:54.238821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b9d8e-720b-4c17-a058-40c367558712 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:54.244992 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b9d8e-720b-4c17-a058-40c367558712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:54.246833 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b9d8e-720b-4c17-a058-40c367558712 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:54.267915 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1089/2198] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:56:54.269173 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003f75e-59ab-4c24-8032-8e72466a91cd admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:54.282650 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003f75e-59ab-4c24-8032-8e72466a91cd admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:54.289602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af7e0c-fab9-469b-b307-5dddccba6967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:54.290782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af7e0c-fab9-469b-b307-5dddccba6967 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:54.291162 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af7e0c-fab9-469b-b307-5dddccba6967 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:54.293344 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af7e0c-fab9-469b-b307-5dddccba6967 admin admin] PATH_INFO: `/v3/projects/a987f7b14d4042bb9c2e2eeac1df7c28/users/eac8f9cdafd243ebbd0d8dc1cafc0c64/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:54.311287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af7e0c-fab9-469b-b307-5dddccba6967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a987f7b14d4042bb9c2e2eeac1df7c28, user_id=eac8f9cdafd243ebbd0d8dc1cafc0c64, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:54.314189 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af7e0c-fab9-469b-b307-5dddccba6967 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:54.329604 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1110/2199] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:54 2022] PUT /identity/v3/projects/a987f7b14d4042bb9c2e2eeac1df7c28/users/eac8f9cdafd243ebbd0d8dc1cafc0c64/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:54.333881 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b9d8e-720b-4c17-a058-40c367558712 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:54.346765 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b9d8e-720b-4c17-a058-40c367558712 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:54.378238 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:54.411394 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238589bc-fc43-44c7-8444-9bd0d940af1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:54.412135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238589bc-fc43-44c7-8444-9bd0d940af1a admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:54.412512 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238589bc-fc43-44c7-8444-9bd0d940af1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:54.412851 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238589bc-fc43-44c7-8444-9bd0d940af1a admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:54.414140 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238589bc-fc43-44c7-8444-9bd0d940af1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:54.415823 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238589bc-fc43-44c7-8444-9bd0d940af1a admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:54.426279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1090/2200] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:54.431068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af7e0c-fab9-469b-b307-5dddccba6967 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:54.441596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af7e0c-fab9-469b-b307-5dddccba6967 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:54.447411 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713b9ce-bb96-434c-adad-0a8f3c48c415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:54.448067 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713b9ce-bb96-434c-adad-0a8f3c48c415 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:54.448375 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713b9ce-bb96-434c-adad-0a8f3c48c415 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:54.448648 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713b9ce-bb96-434c-adad-0a8f3c48c415 admin admin] PATH_INFO: `/v3/projects/a987f7b14d4042bb9c2e2eeac1df7c28/users/eac8f9cdafd243ebbd0d8dc1cafc0c64/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:54.450535 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713b9ce-bb96-434c-adad-0a8f3c48c415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a987f7b14d4042bb9c2e2eeac1df7c28, user_id=eac8f9cdafd243ebbd0d8dc1cafc0c64, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:54.453198 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713b9ce-bb96-434c-adad-0a8f3c48c415 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:54.464876 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1111/2201] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:54 2022] PUT /identity/v3/projects/a987f7b14d4042bb9c2e2eeac1df7c28/users/eac8f9cdafd243ebbd0d8dc1cafc0c64/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:56:54.476518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8524b-a4c5-42d2-8960-22dd4a9eaa18 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:54.476831 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8524b-a4c5-42d2-8960-22dd4a9eaa18 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:54.477108 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8524b-a4c5-42d2-8960-22dd4a9eaa18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:54.505111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-58e8524b-a4c5-42d2-8960-22dd4a9eaa18 None None] MFA Rules not processed for user `eac8f9cdafd243ebbd0d8dc1cafc0c64`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:54.506320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-58e8524b-a4c5-42d2-8960-22dd4a9eaa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:54.574438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1091/2202] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:54.586198 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713b9ce-bb96-434c-adad-0a8f3c48c415 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:54.587946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-2713b9ce-bb96-434c-adad-0a8f3c48c415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:54.646924 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713b9ce-bb96-434c-adad-0a8f3c48c415 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:54.651406 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cc883-8cbd-441c-b47e-0606b4fef770 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:54.652018 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cc883-8cbd-441c-b47e-0606b4fef770 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:54.652333 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cc883-8cbd-441c-b47e-0606b4fef770 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:54.652605 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cc883-8cbd-441c-b47e-0606b4fef770 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:54.709508 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807cc883-8cbd-441c-b47e-0606b4fef770 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:54.711551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807cc883-8cbd-441c-b47e-0606b4fef770 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:54.720432 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1112/2203] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:56:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:56:57.273553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8524b-a4c5-42d2-8960-22dd4a9eaa18 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:57.337153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8524b-a4c5-42d2-8960-22dd4a9eaa18 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:57.341470 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ab2ef-a8ec-487f-bdf7-1fce0167b463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:57.342098 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ab2ef-a8ec-487f-bdf7-1fce0167b463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:57.342373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ab2ef-a8ec-487f-bdf7-1fce0167b463 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:57.342645 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ab2ef-a8ec-487f-bdf7-1fce0167b463 admin admin] PATH_INFO: `/v3/users/e2aaecabea044b1289c722070e77d520` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:57.343747 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ab2ef-a8ec-487f-bdf7-1fce0167b463 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2aaecabea044b1289c722070e77d520)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:57.345700 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ab2ef-a8ec-487f-bdf7-1fce0167b463 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:57.392748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f56ab2ef-a8ec-487f-bdf7-1fce0167b463 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2aaecabea044b1289c722070e77d520'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:57.393403 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f56ab2ef-a8ec-487f-bdf7-1fce0167b463 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2aaecabea044b1289c722070e77d520'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:57.399277 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:57.407549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f56ab2ef-a8ec-487f-bdf7-1fce0167b463 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2aaecabea044b1289c722070e77d520'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:57.415335 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f56ab2ef-a8ec-487f-bdf7-1fce0167b463 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2aaecabea044b1289c722070e77d520'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:57.416046 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f56ab2ef-a8ec-487f-bdf7-1fce0167b463 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2aaecabea044b1289c722070e77d520'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:57.431348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1092/2204] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:56:57 2022] DELETE /identity/v3/users/e2aaecabea044b1289c722070e77d520 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:57.434661 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52c061-a5ca-402a-90df-b4fb3abb412b None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:57.435143 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52c061-a5ca-402a-90df-b4fb3abb412b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:57.435327 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52c061-a5ca-402a-90df-b4fb3abb412b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:57.455515 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-ca52c061-a5ca-402a-90df-b4fb3abb412b None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:57.456842 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ca52c061-a5ca-402a-90df-b4fb3abb412b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:57.491651 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:57.521499 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1113/2205] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:57.531890 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ab2ef-a8ec-487f-bdf7-1fce0167b463 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:57.533365 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-f56ab2ef-a8ec-487f-bdf7-1fce0167b463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:57.594118 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ab2ef-a8ec-487f-bdf7-1fce0167b463 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:57.600119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94827c9f-46fc-4a7c-90ad-a1678204eb51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:57.600752 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94827c9f-46fc-4a7c-90ad-a1678204eb51 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:57.601229 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94827c9f-46fc-4a7c-90ad-a1678204eb51 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:57.601658 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94827c9f-46fc-4a7c-90ad-a1678204eb51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:57.611299 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94827c9f-46fc-4a7c-90ad-a1678204eb51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:57.612787 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94827c9f-46fc-4a7c-90ad-a1678204eb51 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:57.623144 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1093/2206] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:56:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:56:57.968672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52c061-a5ca-402a-90df-b4fb3abb412b None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:57.970151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ca52c061-a5ca-402a-90df-b4fb3abb412b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:57.983801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52c061-a5ca-402a-90df-b4fb3abb412b None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:57.991253 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee878d8-c679-4ee3-9b26-933948d45c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:57.992013 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee878d8-c679-4ee3-9b26-933948d45c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:57.992428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee878d8-c679-4ee3-9b26-933948d45c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:57.992790 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee878d8-c679-4ee3-9b26-933948d45c7c admin admin] PATH_INFO: `/v3/projects/f77a2a1f5bd4485f8f97470774c47713` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:57.994098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee878d8-c679-4ee3-9b26-933948d45c7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f77a2a1f5bd4485f8f97470774c47713)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:57.996073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee878d8-c679-4ee3-9b26-933948d45c7c admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:58.051172 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-eee878d8-c679-4ee3-9b26-933948d45c7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f77a2a1f5bd4485f8f97470774c47713', 'request_id': 'req-eee878d8-c679-4ee3-9b26-933948d45c7c'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:58.066050 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-eee878d8-c679-4ee3-9b26-933948d45c7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f77a2a1f5bd4485f8f97470774c47713', 'request_id': 'req-eee878d8-c679-4ee3-9b26-933948d45c7c'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:58.075423 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-eee878d8-c679-4ee3-9b26-933948d45c7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f77a2a1f5bd4485f8f97470774c47713', 'request_id': 'req-eee878d8-c679-4ee3-9b26-933948d45c7c'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:58.087923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-eee878d8-c679-4ee3-9b26-933948d45c7c admin admin] The token cache is being invalidate because project f77a2a1f5bd4485f8f97470774c47713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:56:58.088250 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-eee878d8-c679-4ee3-9b26-933948d45c7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:58.088927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-eee878d8-c679-4ee3-9b26-933948d45c7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:58.090438 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1114/2207] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:56:57 2022] DELETE /identity/v3/projects/f77a2a1f5bd4485f8f97470774c47713 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:58.094466 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94827c9f-46fc-4a7c-90ad-a1678204eb51 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:58.106168 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94827c9f-46fc-4a7c-90ad-a1678204eb51 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:58.166725 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a983407-2174-4157-9978-e5506c390f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:58.167693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a983407-2174-4157-9978-e5506c390f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:58.168220 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a983407-2174-4157-9978-e5506c390f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:58.168670 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a983407-2174-4157-9978-e5506c390f2c admin admin] PATH_INFO: `/v3/users/eac8f9cdafd243ebbd0d8dc1cafc0c64` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:58.169989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a983407-2174-4157-9978-e5506c390f2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eac8f9cdafd243ebbd0d8dc1cafc0c64)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:58.172159 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a983407-2174-4157-9978-e5506c390f2c admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:58.231942 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2a983407-2174-4157-9978-e5506c390f2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eac8f9cdafd243ebbd0d8dc1cafc0c64'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:58.232884 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2a983407-2174-4157-9978-e5506c390f2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eac8f9cdafd243ebbd0d8dc1cafc0c64'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:58.248968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2a983407-2174-4157-9978-e5506c390f2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eac8f9cdafd243ebbd0d8dc1cafc0c64'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:58.261111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2a983407-2174-4157-9978-e5506c390f2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eac8f9cdafd243ebbd0d8dc1cafc0c64'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:58.262660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2a983407-2174-4157-9978-e5506c390f2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eac8f9cdafd243ebbd0d8dc1cafc0c64'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:58.267259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:58.285398 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1094/2208] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:56:58 2022] DELETE /identity/v3/users/eac8f9cdafd243ebbd0d8dc1cafc0c64 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:58.589119 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee878d8-c679-4ee3-9b26-933948d45c7c admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:58.590376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-eee878d8-c679-4ee3-9b26-933948d45c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:58.604669 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee878d8-c679-4ee3-9b26-933948d45c7c admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:58.664666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae524e-cbc5-481d-8999-749bbd30994b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:58.665310 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae524e-cbc5-481d-8999-749bbd30994b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:58.665640 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae524e-cbc5-481d-8999-749bbd30994b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:58.665937 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae524e-cbc5-481d-8999-749bbd30994b admin admin] PATH_INFO: `/v3/projects/a987f7b14d4042bb9c2e2eeac1df7c28` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:58.667107 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae524e-cbc5-481d-8999-749bbd30994b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a987f7b14d4042bb9c2e2eeac1df7c28)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:58.669040 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae524e-cbc5-481d-8999-749bbd30994b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:58.720724 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-71ae524e-cbc5-481d-8999-749bbd30994b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a987f7b14d4042bb9c2e2eeac1df7c28', 'request_id': 'req-71ae524e-cbc5-481d-8999-749bbd30994b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:58.729520 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-71ae524e-cbc5-481d-8999-749bbd30994b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a987f7b14d4042bb9c2e2eeac1df7c28', 'request_id': 'req-71ae524e-cbc5-481d-8999-749bbd30994b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:58.734512 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:58.736070 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-71ae524e-cbc5-481d-8999-749bbd30994b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a987f7b14d4042bb9c2e2eeac1df7c28', 'request_id': 'req-71ae524e-cbc5-481d-8999-749bbd30994b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:58.750415 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-71ae524e-cbc5-481d-8999-749bbd30994b admin admin] The token cache is being invalidate because project a987f7b14d4042bb9c2e2eeac1df7c28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:56:58.750692 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-71ae524e-cbc5-481d-8999-749bbd30994b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:58.751369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-71ae524e-cbc5-481d-8999-749bbd30994b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:56:58.752883 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1115/2209] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:56:58 2022] DELETE /identity/v3/projects/a987f7b14d4042bb9c2e2eeac1df7c28 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:58.758503 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00973fa-3ff8-4adf-9d75-d141fab3d4aa None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:58.758872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00973fa-3ff8-4adf-9d75-d141fab3d4aa None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:58.759210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00973fa-3ff8-4adf-9d75-d141fab3d4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:58.779711 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-c00973fa-3ff8-4adf-9d75-d141fab3d4aa None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:58.780943 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c00973fa-3ff8-4adf-9d75-d141fab3d4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:58.839502 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1095/2210] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:56:58.847303 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd10021-9415-4d65-bc04-96a203fb3a24 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:58.847610 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd10021-9415-4d65-bc04-96a203fb3a24 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:58.848058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd10021-9415-4d65-bc04-96a203fb3a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:58.868627 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-bcd10021-9415-4d65-bc04-96a203fb3a24 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:58.869908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-bcd10021-9415-4d65-bc04-96a203fb3a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:58.877890 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1116/2211] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:58.882246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00973fa-3ff8-4adf-9d75-d141fab3d4aa None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:58.891931 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00973fa-3ff8-4adf-9d75-d141fab3d4aa None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:58.896729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91ed44-acf2-4315-80df-aebca695122b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:58.897333 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91ed44-acf2-4315-80df-aebca695122b admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:58.897607 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91ed44-acf2-4315-80df-aebca695122b admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:58.898366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91ed44-acf2-4315-80df-aebca695122b admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:58.899528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91ed44-acf2-4315-80df-aebca695122b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:56:58.899828 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91ed44-acf2-4315-80df-aebca695122b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:58.901475 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91ed44-acf2-4315-80df-aebca695122b admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:58.915753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1096/2212] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:56:58.920212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd10021-9415-4d65-bc04-96a203fb3a24 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:58.929585 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd10021-9415-4d65-bc04-96a203fb3a24 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:58.934324 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55aa39b-21dd-4925-8c08-dabc3b3647ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:58.934932 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55aa39b-21dd-4925-8c08-dabc3b3647ac admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:58.935215 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55aa39b-21dd-4925-8c08-dabc3b3647ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:58.935489 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55aa39b-21dd-4925-8c08-dabc3b3647ac admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:58.936143 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55aa39b-21dd-4925-8c08-dabc3b3647ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:58.938077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55aa39b-21dd-4925-8c08-dabc3b3647ac admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:58.970911 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1117/2213] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:56:58 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:58.971947 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91ed44-acf2-4315-80df-aebca695122b admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:58.982100 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91ed44-acf2-4315-80df-aebca695122b admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:59.037933 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18b013-cccf-41f2-8ce8-76159137a064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:59.038573 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18b013-cccf-41f2-8ce8-76159137a064 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:59.038919 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18b013-cccf-41f2-8ce8-76159137a064 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:59.039209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18b013-cccf-41f2-8ce8-76159137a064 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:59.039842 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18b013-cccf-41f2-8ce8-76159137a064 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:59.041806 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18b013-cccf-41f2-8ce8-76159137a064 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:59.062702 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1097/2214] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:56:58 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:59.066986 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55aa39b-21dd-4925-8c08-dabc3b3647ac admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:59.077794 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55aa39b-21dd-4925-8c08-dabc3b3647ac admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:59.082343 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da9819-b479-4265-b821-4f8f31dc775b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:59.083191 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da9819-b479-4265-b821-4f8f31dc775b admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:59.083469 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da9819-b479-4265-b821-4f8f31dc775b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:59.083740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da9819-b479-4265-b821-4f8f31dc775b admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:59.084411 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da9819-b479-4265-b821-4f8f31dc775b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:59.086747 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da9819-b479-4265-b821-4f8f31dc775b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:59.096583 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1118/2215] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:59.100447 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18b013-cccf-41f2-8ce8-76159137a064 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:59.110575 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18b013-cccf-41f2-8ce8-76159137a064 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:59.115212 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15556c4b-2539-4553-8e4c-0b79aef6033f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:59.115799 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15556c4b-2539-4553-8e4c-0b79aef6033f admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:59.116110 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15556c4b-2539-4553-8e4c-0b79aef6033f admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:59.116384 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15556c4b-2539-4553-8e4c-0b79aef6033f admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:59.117257 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15556c4b-2539-4553-8e4c-0b79aef6033f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:59.118876 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15556c4b-2539-4553-8e4c-0b79aef6033f admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:59.129351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1098/2216] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:59.133915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da9819-b479-4265-b821-4f8f31dc775b admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:59.143715 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da9819-b479-4265-b821-4f8f31dc775b admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:59.149042 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34903aa-1b7e-4169-8f02-668199c29783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:59.149735 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34903aa-1b7e-4169-8f02-668199c29783 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:59.150072 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34903aa-1b7e-4169-8f02-668199c29783 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:59.150427 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34903aa-1b7e-4169-8f02-668199c29783 admin admin] PATH_INFO: `/v3/projects/7b685e14606e40779228e810fa2a7559/users/afd1121bf2084f09bb868c661b5fcb14/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:59.165152 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34903aa-1b7e-4169-8f02-668199c29783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b685e14606e40779228e810fa2a7559, user_id=afd1121bf2084f09bb868c661b5fcb14, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:59.167809 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34903aa-1b7e-4169-8f02-668199c29783 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:59.180770 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1119/2217] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:59 2022] PUT /identity/v3/projects/7b685e14606e40779228e810fa2a7559/users/afd1121bf2084f09bb868c661b5fcb14/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:59.190851 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784011a4-b877-4700-b035-b32eb7da9403 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:59.191137 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784011a4-b877-4700-b035-b32eb7da9403 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:59.191436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784011a4-b877-4700-b035-b32eb7da9403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:59.219989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-784011a4-b877-4700-b035-b32eb7da9403 None None] MFA Rules not processed for user `afd1121bf2084f09bb868c661b5fcb14`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:59.221336 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-784011a4-b877-4700-b035-b32eb7da9403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:59.267294 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:59.286392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1099/2218] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:59.291310 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34903aa-1b7e-4169-8f02-668199c29783 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:59.353290 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34903aa-1b7e-4169-8f02-668199c29783 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:59.357857 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ea324-95e5-4414-ab20-654a8fe777f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:59.358461 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ea324-95e5-4414-ab20-654a8fe777f4 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:59.358739 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ea324-95e5-4414-ab20-654a8fe777f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:59.359010 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ea324-95e5-4414-ab20-654a8fe777f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:59.359628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ea324-95e5-4414-ab20-654a8fe777f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:59.361532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ea324-95e5-4414-ab20-654a8fe777f4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:59.389823 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1120/2219] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:56:59 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:59.393825 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784011a4-b877-4700-b035-b32eb7da9403 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:59.404569 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784011a4-b877-4700-b035-b32eb7da9403 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:59.460416 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15633b9b-1c39-4de4-b032-b4045072a88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:59.461022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15633b9b-1c39-4de4-b032-b4045072a88e admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:59.461303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15633b9b-1c39-4de4-b032-b4045072a88e admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:59.461573 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15633b9b-1c39-4de4-b032-b4045072a88e admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:59.462205 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15633b9b-1c39-4de4-b032-b4045072a88e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:59.464148 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15633b9b-1c39-4de4-b032-b4045072a88e admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:59.484929 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1100/2220] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:56:59 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:59.488829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ea324-95e5-4414-ab20-654a8fe777f4 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:59.500198 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ea324-95e5-4414-ab20-654a8fe777f4 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:59.505087 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcd55b-62c2-4761-b27b-1c3f3e00ff31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:59.505704 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcd55b-62c2-4761-b27b-1c3f3e00ff31 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:59.505989 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcd55b-62c2-4761-b27b-1c3f3e00ff31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:59.506270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcd55b-62c2-4761-b27b-1c3f3e00ff31 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:59.506973 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffcd55b-62c2-4761-b27b-1c3f3e00ff31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:59.509268 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffcd55b-62c2-4761-b27b-1c3f3e00ff31 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:59.520016 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1121/2221] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:59.524142 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15633b9b-1c39-4de4-b032-b4045072a88e admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:59.533934 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15633b9b-1c39-4de4-b032-b4045072a88e admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:59.538645 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08441584-bd5b-40da-9090-3bb84ec5fb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:59.539219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08441584-bd5b-40da-9090-3bb84ec5fb48 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:59.539527 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08441584-bd5b-40da-9090-3bb84ec5fb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:59.539788 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08441584-bd5b-40da-9090-3bb84ec5fb48 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:59.540540 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08441584-bd5b-40da-9090-3bb84ec5fb48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:59.542140 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08441584-bd5b-40da-9090-3bb84ec5fb48 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:59.551918 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1101/2222] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:59.558196 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcd55b-62c2-4761-b27b-1c3f3e00ff31 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:59.568420 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcd55b-62c2-4761-b27b-1c3f3e00ff31 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:59.573606 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd3d52-a8c0-43bf-aad0-1e27b9f47713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:59.574273 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd3d52-a8c0-43bf-aad0-1e27b9f47713 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:59.574553 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd3d52-a8c0-43bf-aad0-1e27b9f47713 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:59.574828 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd3d52-a8c0-43bf-aad0-1e27b9f47713 admin admin] PATH_INFO: `/v3/projects/59e634fa58c446b4b8b399d6dd26fbe9/users/8c83108c27eb4a7aa25ecae30e4b083f/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:59.590540 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd3d52-a8c0-43bf-aad0-1e27b9f47713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59e634fa58c446b4b8b399d6dd26fbe9, user_id=8c83108c27eb4a7aa25ecae30e4b083f, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:59.593244 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd3d52-a8c0-43bf-aad0-1e27b9f47713 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:59.607626 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1122/2223] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:59 2022] PUT /identity/v3/projects/59e634fa58c446b4b8b399d6dd26fbe9/users/8c83108c27eb4a7aa25ecae30e4b083f/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:56:59.618434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7077b-e42d-4329-afee-85eb036ddc36 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:59.618724 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7077b-e42d-4329-afee-85eb036ddc36 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:59.619001 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7077b-e42d-4329-afee-85eb036ddc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:59.647987 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-38d7077b-e42d-4329-afee-85eb036ddc36 None None] MFA Rules not processed for user `8c83108c27eb4a7aa25ecae30e4b083f`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:56:59.649585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-38d7077b-e42d-4329-afee-85eb036ddc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:56:59.713623 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1102/2224] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:56:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:56:59.718141 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd3d52-a8c0-43bf-aad0-1e27b9f47713 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:59.759882 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:59.781529 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd3d52-a8c0-43bf-aad0-1e27b9f47713 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:59.786431 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbd917-f612-4847-9fa4-1f5713ca6b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:59.787043 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbd917-f612-4847-9fa4-1f5713ca6b15 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:59.787322 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbd917-f612-4847-9fa4-1f5713ca6b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:59.787596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbd917-f612-4847-9fa4-1f5713ca6b15 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:59.788269 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cbd917-f612-4847-9fa4-1f5713ca6b15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:59.790844 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cbd917-f612-4847-9fa4-1f5713ca6b15 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:59.817771 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1123/2225] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:56:59 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:59.822119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7077b-e42d-4329-afee-85eb036ddc36 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:59.833268 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7077b-e42d-4329-afee-85eb036ddc36 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:59.885903 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:56:59.894094 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a23432-94de-410b-b2e7-0502cc34b962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:59.894965 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a23432-94de-410b-b2e7-0502cc34b962 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:59.895533 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a23432-94de-410b-b2e7-0502cc34b962 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:59.895833 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a23432-94de-410b-b2e7-0502cc34b962 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:59.896526 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a23432-94de-410b-b2e7-0502cc34b962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:59.898662 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a23432-94de-410b-b2e7-0502cc34b962 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:59.922769 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1103/2226] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:56:59 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:56:59.926326 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbd917-f612-4847-9fa4-1f5713ca6b15 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:59.937710 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbd917-f612-4847-9fa4-1f5713ca6b15 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:59.942307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cbeda-46a4-4f05-80f4-4a69ed19d033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:59.942892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cbeda-46a4-4f05-80f4-4a69ed19d033 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:59.943169 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cbeda-46a4-4f05-80f4-4a69ed19d033 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:59.943439 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cbeda-46a4-4f05-80f4-4a69ed19d033 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:59.944068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cbeda-46a4-4f05-80f4-4a69ed19d033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:59.946242 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cbeda-46a4-4f05-80f4-4a69ed19d033 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:56:59.956733 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1124/2227] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:56:59.961668 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a23432-94de-410b-b2e7-0502cc34b962 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:56:59.971475 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a23432-94de-410b-b2e7-0502cc34b962 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:56:59.977009 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da6bfb-83ea-49be-823b-643b0d6ba920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:56:59.977535 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da6bfb-83ea-49be-823b-643b0d6ba920 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:56:59.977811 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da6bfb-83ea-49be-823b-643b0d6ba920 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:56:59.978079 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da6bfb-83ea-49be-823b-643b0d6ba920 admin admin] PATH_INFO: `/v3/projects/e363beac935f492d95564c74beb6eb6e/users/8a3f11f50a934103b0d7de00b6008345/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:56:59.993369 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da6bfb-83ea-49be-823b-643b0d6ba920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e363beac935f492d95564c74beb6eb6e, user_id=8a3f11f50a934103b0d7de00b6008345, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:56:59.996012 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da6bfb-83ea-49be-823b-643b0d6ba920 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:57:00.008665 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1104/2228] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:56:59 2022] PUT /identity/v3/projects/e363beac935f492d95564c74beb6eb6e/users/8a3f11f50a934103b0d7de00b6008345/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:57:00.012787 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cbeda-46a4-4f05-80f4-4a69ed19d033 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:57:00.022652 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cbeda-46a4-4f05-80f4-4a69ed19d033 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:57:00.079873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510f035-9256-4bc4-9bf3-5024f3ec10af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:57:00.080503 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510f035-9256-4bc4-9bf3-5024f3ec10af admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:00.080782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510f035-9256-4bc4-9bf3-5024f3ec10af admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:00.081055 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510f035-9256-4bc4-9bf3-5024f3ec10af admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:00.081680 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510f035-9256-4bc4-9bf3-5024f3ec10af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:57:00.083547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510f035-9256-4bc4-9bf3-5024f3ec10af admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:57:00.093839 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1125/2229] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:57:00.098181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da6bfb-83ea-49be-823b-643b0d6ba920 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:57:00.109392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da6bfb-83ea-49be-823b-643b0d6ba920 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:57:00.114158 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7627ead-1570-47b3-bb18-d9e7e321b87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:57:00.114801 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7627ead-1570-47b3-bb18-d9e7e321b87f admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:00.115086 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7627ead-1570-47b3-bb18-d9e7e321b87f admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:00.115362 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7627ead-1570-47b3-bb18-d9e7e321b87f admin admin] PATH_INFO: `/v3/projects/e363beac935f492d95564c74beb6eb6e/users/8a3f11f50a934103b0d7de00b6008345/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:00.117407 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7627ead-1570-47b3-bb18-d9e7e321b87f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e363beac935f492d95564c74beb6eb6e, user_id=8a3f11f50a934103b0d7de00b6008345, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:57:00.120074 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7627ead-1570-47b3-bb18-d9e7e321b87f admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:57:00.130136 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1105/2230] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:57:00 2022] PUT /identity/v3/projects/e363beac935f492d95564c74beb6eb6e/users/8a3f11f50a934103b0d7de00b6008345/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:57:00.140612 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c19eb-d624-45b3-9ea9-c019057c0c01 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:00.140897 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c19eb-d624-45b3-9ea9-c019057c0c01 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:00.141171 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c19eb-d624-45b3-9ea9-c019057c0c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:00.169727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-3d3c19eb-d624-45b3-9ea9-c019057c0c01 None None] MFA Rules not processed for user `8a3f11f50a934103b0d7de00b6008345`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:57:00.170936 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-3d3c19eb-d624-45b3-9ea9-c019057c0c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:57:00.237822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1126/2231] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:57:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:57:00.249579 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7627ead-1570-47b3-bb18-d9e7e321b87f admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:57:00.251320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e7627ead-1570-47b3-bb18-d9e7e321b87f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:57:00.310154 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7627ead-1570-47b3-bb18-d9e7e321b87f admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:57:00.313821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b589c-4ed3-423d-9a2e-8951c63ce5ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:57:00.314429 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b589c-4ed3-423d-9a2e-8951c63ce5ba service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:00.314710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b589c-4ed3-423d-9a2e-8951c63ce5ba service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:00.315012 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b589c-4ed3-423d-9a2e-8951c63ce5ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:00.375534 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b589c-4ed3-423d-9a2e-8951c63ce5ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:57:00.377021 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b589c-4ed3-423d-9a2e-8951c63ce5ba service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:57:00.386118 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1106/2232] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:57:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:57:01.890600 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c19eb-d624-45b3-9ea9-c019057c0c01 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:57:01.892127 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-3d3c19eb-d624-45b3-9ea9-c019057c0c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:57:01.904549 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c19eb-d624-45b3-9ea9-c019057c0c01 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:57:01.908230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ed71e-7be1-4737-a858-ab1c5da317ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:57:01.908858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ed71e-7be1-4737-a858-ab1c5da317ce service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:01.909137 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ed71e-7be1-4737-a858-ab1c5da317ce service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:01.909411 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ed71e-7be1-4737-a858-ab1c5da317ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:01.941174 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:57:01.968655 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ed71e-7be1-4737-a858-ab1c5da317ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:57:01.970119 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ed71e-7be1-4737-a858-ab1c5da317ce service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:57:01.979691 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1127/2233] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:57:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:57:02.510705 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b589c-4ed3-423d-9a2e-8951c63ce5ba service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:57:02.519276 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b589c-4ed3-423d-9a2e-8951c63ce5ba service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:57:02.523451 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d68228-ddf0-46cd-a4e1-0c4609178de8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:57:02.524075 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d68228-ddf0-46cd-a4e1-0c4609178de8 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:02.524378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d68228-ddf0-46cd-a4e1-0c4609178de8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:02.524652 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d68228-ddf0-46cd-a4e1-0c4609178de8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:02.534493 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d68228-ddf0-46cd-a4e1-0c4609178de8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:57:02.535962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d68228-ddf0-46cd-a4e1-0c4609178de8 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:57:02.546438 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1107/2234] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:57:02 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:57:06.426443 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ed71e-7be1-4737-a858-ab1c5da317ce service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:57:06.489330 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ed71e-7be1-4737-a858-ab1c5da317ce service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:57:06.494204 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c0548b-8f8f-4be2-b8a7-57d7d3425d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:57:06.494824 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0548b-8f8f-4be2-b8a7-57d7d3425d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:06.495103 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0548b-8f8f-4be2-b8a7-57d7d3425d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:06.495376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0548b-8f8f-4be2-b8a7-57d7d3425d16 admin admin] PATH_INFO: `/v3/users/afd1121bf2084f09bb868c661b5fcb14` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:06.496499 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c0548b-8f8f-4be2-b8a7-57d7d3425d16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afd1121bf2084f09bb868c661b5fcb14)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:57:06.498409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c0548b-8f8f-4be2-b8a7-57d7d3425d16 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:57:06.545356 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-06c0548b-8f8f-4be2-b8a7-57d7d3425d16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afd1121bf2084f09bb868c661b5fcb14'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:06.546006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-06c0548b-8f8f-4be2-b8a7-57d7d3425d16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afd1121bf2084f09bb868c661b5fcb14'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:06.558428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-06c0548b-8f8f-4be2-b8a7-57d7d3425d16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afd1121bf2084f09bb868c661b5fcb14'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:06.565119 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-06c0548b-8f8f-4be2-b8a7-57d7d3425d16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afd1121bf2084f09bb868c661b5fcb14'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:06.573181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-06c0548b-8f8f-4be2-b8a7-57d7d3425d16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afd1121bf2084f09bb868c661b5fcb14'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:06.579567 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1128/2235] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:57:06 2022] DELETE /identity/v3/users/afd1121bf2084f09bb868c661b5fcb14 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:57:06.584651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457fa0e-7336-4fad-af0a-6d041820052c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:06.585166 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457fa0e-7336-4fad-af0a-6d041820052c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:06.585298 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457fa0e-7336-4fad-af0a-6d041820052c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:06.604823 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-a457fa0e-7336-4fad-af0a-6d041820052c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:57:06.606015 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a457fa0e-7336-4fad-af0a-6d041820052c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:57:06.651420 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:57:06.666739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1108/2236] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:57:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:57:06.676413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c0548b-8f8f-4be2-b8a7-57d7d3425d16 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:57:06.677808 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-06c0548b-8f8f-4be2-b8a7-57d7d3425d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:57:06.736858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c0548b-8f8f-4be2-b8a7-57d7d3425d16 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:57:06.740433 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050c997-2e85-4041-8742-ad6aef580c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:57:06.741035 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050c997-2e85-4041-8742-ad6aef580c4b service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:06.741310 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050c997-2e85-4041-8742-ad6aef580c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:06.741577 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050c997-2e85-4041-8742-ad6aef580c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:06.747568 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:57:06.752329 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050c997-2e85-4041-8742-ad6aef580c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:57:06.754084 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050c997-2e85-4041-8742-ad6aef580c4b service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:57:06.765277 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1129/2237] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:57:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:57:07.047386 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457fa0e-7336-4fad-af0a-6d041820052c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:57:07.048901 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a457fa0e-7336-4fad-af0a-6d041820052c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:57:07.061367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457fa0e-7336-4fad-af0a-6d041820052c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:57:07.066387 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac766aa-0615-4ea1-b921-44d1d24fb9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:57:07.067101 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac766aa-0615-4ea1-b921-44d1d24fb9b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:07.067481 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac766aa-0615-4ea1-b921-44d1d24fb9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:07.067857 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac766aa-0615-4ea1-b921-44d1d24fb9b2 admin admin] PATH_INFO: `/v3/projects/7b685e14606e40779228e810fa2a7559` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:07.069242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac766aa-0615-4ea1-b921-44d1d24fb9b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b685e14606e40779228e810fa2a7559)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:57:07.072103 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac766aa-0615-4ea1-b921-44d1d24fb9b2 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:57:07.121092 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-bac766aa-0615-4ea1-b921-44d1d24fb9b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b685e14606e40779228e810fa2a7559', 'request_id': 'req-bac766aa-0615-4ea1-b921-44d1d24fb9b2'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.137399 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-bac766aa-0615-4ea1-b921-44d1d24fb9b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b685e14606e40779228e810fa2a7559', 'request_id': 'req-bac766aa-0615-4ea1-b921-44d1d24fb9b2'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.144076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-bac766aa-0615-4ea1-b921-44d1d24fb9b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b685e14606e40779228e810fa2a7559', 'request_id': 'req-bac766aa-0615-4ea1-b921-44d1d24fb9b2'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.154603 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-bac766aa-0615-4ea1-b921-44d1d24fb9b2 admin admin] The token cache is being invalidate because project 7b685e14606e40779228e810fa2a7559 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:57:07.154895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-bac766aa-0615-4ea1-b921-44d1d24fb9b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.155536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-bac766aa-0615-4ea1-b921-44d1d24fb9b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.157069 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1109/2238] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:57:07 2022] DELETE /identity/v3/projects/7b685e14606e40779228e810fa2a7559 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:57:07.161015 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050c997-2e85-4041-8742-ad6aef580c4b service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:57:07.171637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050c997-2e85-4041-8742-ad6aef580c4b service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:57:07.229004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d19e2-7380-41a8-94db-2a4c23ffa594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:57:07.229633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d19e2-7380-41a8-94db-2a4c23ffa594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:07.229919 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d19e2-7380-41a8-94db-2a4c23ffa594 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:07.230191 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d19e2-7380-41a8-94db-2a4c23ffa594 admin admin] PATH_INFO: `/v3/users/8c83108c27eb4a7aa25ecae30e4b083f` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:07.231331 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d19e2-7380-41a8-94db-2a4c23ffa594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c83108c27eb4a7aa25ecae30e4b083f)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:57:07.233302 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d19e2-7380-41a8-94db-2a4c23ffa594 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:57:07.279092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-2a5d19e2-7380-41a8-94db-2a4c23ffa594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c83108c27eb4a7aa25ecae30e4b083f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.279729 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-2a5d19e2-7380-41a8-94db-2a4c23ffa594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c83108c27eb4a7aa25ecae30e4b083f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.292465 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-2a5d19e2-7380-41a8-94db-2a4c23ffa594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c83108c27eb4a7aa25ecae30e4b083f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.299221 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-2a5d19e2-7380-41a8-94db-2a4c23ffa594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c83108c27eb4a7aa25ecae30e4b083f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.307456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-2a5d19e2-7380-41a8-94db-2a4c23ffa594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c83108c27eb4a7aa25ecae30e4b083f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.314568 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1130/2239] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:57:07 2022] DELETE /identity/v3/users/8c83108c27eb4a7aa25ecae30e4b083f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:57:07.584914 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac766aa-0615-4ea1-b921-44d1d24fb9b2 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:57:07.587311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-bac766aa-0615-4ea1-b921-44d1d24fb9b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:57:07.608352 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac766aa-0615-4ea1-b921-44d1d24fb9b2 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:57:07.671856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47640de4-b8a4-4daf-9fd1-ef098a495a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:57:07.672546 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47640de4-b8a4-4daf-9fd1-ef098a495a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:07.672830 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47640de4-b8a4-4daf-9fd1-ef098a495a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:07.673106 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47640de4-b8a4-4daf-9fd1-ef098a495a1a admin admin] PATH_INFO: `/v3/projects/59e634fa58c446b4b8b399d6dd26fbe9` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:07.674228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47640de4-b8a4-4daf-9fd1-ef098a495a1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=59e634fa58c446b4b8b399d6dd26fbe9)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:57:07.676189 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47640de4-b8a4-4daf-9fd1-ef098a495a1a admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:57:07.728738 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-47640de4-b8a4-4daf-9fd1-ef098a495a1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59e634fa58c446b4b8b399d6dd26fbe9', 'request_id': 'req-47640de4-b8a4-4daf-9fd1-ef098a495a1a'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.737185 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-47640de4-b8a4-4daf-9fd1-ef098a495a1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59e634fa58c446b4b8b399d6dd26fbe9', 'request_id': 'req-47640de4-b8a4-4daf-9fd1-ef098a495a1a'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.743918 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-47640de4-b8a4-4daf-9fd1-ef098a495a1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59e634fa58c446b4b8b399d6dd26fbe9', 'request_id': 'req-47640de4-b8a4-4daf-9fd1-ef098a495a1a'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.755229 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-47640de4-b8a4-4daf-9fd1-ef098a495a1a admin admin] The token cache is being invalidate because project 59e634fa58c446b4b8b399d6dd26fbe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:57:07.755531 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-47640de4-b8a4-4daf-9fd1-ef098a495a1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.756225 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-47640de4-b8a4-4daf-9fd1-ef098a495a1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.761328 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1110/2240] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:57:07 2022] DELETE /identity/v3/projects/59e634fa58c446b4b8b399d6dd26fbe9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:57:07.762282 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d19e2-7380-41a8-94db-2a4c23ffa594 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:57:07.763720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-2a5d19e2-7380-41a8-94db-2a4c23ffa594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:57:07.777640 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d19e2-7380-41a8-94db-2a4c23ffa594 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:57:07.846202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ddf44-f17a-47ef-a971-afd6b909b404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:57:07.847089 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ddf44-f17a-47ef-a971-afd6b909b404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:07.847539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ddf44-f17a-47ef-a971-afd6b909b404 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:07.847835 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ddf44-f17a-47ef-a971-afd6b909b404 admin admin] PATH_INFO: `/v3/users/8a3f11f50a934103b0d7de00b6008345` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:07.849050 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ddf44-f17a-47ef-a971-afd6b909b404 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a3f11f50a934103b0d7de00b6008345)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:57:07.851350 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ddf44-f17a-47ef-a971-afd6b909b404 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:57:07.870287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:57:07.913917 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-de8ddf44-f17a-47ef-a971-afd6b909b404 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a3f11f50a934103b0d7de00b6008345'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.915173 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-de8ddf44-f17a-47ef-a971-afd6b909b404 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a3f11f50a934103b0d7de00b6008345'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.927863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-de8ddf44-f17a-47ef-a971-afd6b909b404 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a3f11f50a934103b0d7de00b6008345'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.935507 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-de8ddf44-f17a-47ef-a971-afd6b909b404 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a3f11f50a934103b0d7de00b6008345'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.943870 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-de8ddf44-f17a-47ef-a971-afd6b909b404 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a3f11f50a934103b0d7de00b6008345'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:07.951161 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1131/2241] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:57:07 2022] DELETE /identity/v3/users/8a3f11f50a934103b0d7de00b6008345 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:57:08.218733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47640de4-b8a4-4daf-9fd1-ef098a495a1a admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:57:08.220346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-47640de4-b8a4-4daf-9fd1-ef098a495a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:57:08.231017 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:57:08.236228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47640de4-b8a4-4daf-9fd1-ef098a495a1a admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:57:08.296632 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09951dd5-579d-413c-af72-9c11277e45c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:57:08.297291 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09951dd5-579d-413c-af72-9c11277e45c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:08.297571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09951dd5-579d-413c-af72-9c11277e45c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:08.297846 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09951dd5-579d-413c-af72-9c11277e45c2 admin admin] PATH_INFO: `/v3/projects/e363beac935f492d95564c74beb6eb6e` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:08.298944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09951dd5-579d-413c-af72-9c11277e45c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e363beac935f492d95564c74beb6eb6e)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:57:08.300918 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09951dd5-579d-413c-af72-9c11277e45c2 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:57:08.348544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-09951dd5-579d-413c-af72-9c11277e45c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e363beac935f492d95564c74beb6eb6e', 'request_id': 'req-09951dd5-579d-413c-af72-9c11277e45c2'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:08.356046 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-09951dd5-579d-413c-af72-9c11277e45c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e363beac935f492d95564c74beb6eb6e', 'request_id': 'req-09951dd5-579d-413c-af72-9c11277e45c2'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:08.362289 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-09951dd5-579d-413c-af72-9c11277e45c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e363beac935f492d95564c74beb6eb6e', 'request_id': 'req-09951dd5-579d-413c-af72-9c11277e45c2'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:08.372873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-09951dd5-579d-413c-af72-9c11277e45c2 admin admin] The token cache is being invalidate because project e363beac935f492d95564c74beb6eb6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:57:08.373160 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-09951dd5-579d-413c-af72-9c11277e45c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:08.373784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-09951dd5-579d-413c-af72-9c11277e45c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:57:08.375244 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1111/2242] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:57:08 2022] DELETE /identity/v3/projects/e363beac935f492d95564c74beb6eb6e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:57:17.947040 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=21,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:57:18.370011 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=49,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:57:25.837586 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae59d8-843f-4bb2-ad29-c04ce2cad5a3 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:25.838039 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae59d8-843f-4bb2-ad29-c04ce2cad5a3 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:25.838195 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae59d8-843f-4bb2-ad29-c04ce2cad5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:25.868213 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-b7ae59d8-843f-4bb2-ad29-c04ce2cad5a3 None None] MFA Rules not processed for user `8a91d3b9c4834d729211ec367e235889`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:57:25.869887 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-b7ae59d8-843f-4bb2-ad29-c04ce2cad5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:57:25.931820 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1132/2243] 10.222.0.36 () {58 vars in 1009 bytes} [Sun Oct 30 00:57:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:57:25.983415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09951dd5-579d-413c-af72-9c11277e45c2 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:57:25.984904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-09951dd5-579d-413c-af72-9c11277e45c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:57:26.054939 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09951dd5-579d-413c-af72-9c11277e45c2 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:57:26.060240 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c70e02-c991-4ed3-9d54-fd4720272fbb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f4c779b8422432fb06cdf996841b9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:57:26.060852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c70e02-c991-4ed3-9d54-fd4720272fbb service swift] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:26.061131 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c70e02-c991-4ed3-9d54-fd4720272fbb service swift] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:26.061405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c70e02-c991-4ed3-9d54-fd4720272fbb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:26.069665 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c70e02-c991-4ed3-9d54-fd4720272fbb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:57:26.071256 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c70e02-c991-4ed3-9d54-fd4720272fbb service swift] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:57:26.080232 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1112/2244] 10.222.0.36 () {60 vars in 1384 bytes} [Sun Oct 30 00:57:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 00:57:35.925118 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=29 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:57:36.076985 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=32 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:57:39.950630 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c5734-e203-4755-bce5-4f838b6e1dfd None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:39.951256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c5734-e203-4755-bce5-4f838b6e1dfd None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:39.951256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c5734-e203-4755-bce5-4f838b6e1dfd None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:39.952565 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1133/2245] 10.222.0.36 () {54 vars in 921 bytes} [Sun Oct 30 00:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:57:39.958399 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcaa63-08df-4d0d-a550-0dc169d6e64e None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:39.958877 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcaa63-08df-4d0d-a550-0dc169d6e64e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:39.959102 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcaa63-08df-4d0d-a550-0dc169d6e64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:39.981218 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-a8fcaa63-08df-4d0d-a550-0dc169d6e64e None None] MFA Rules not processed for user `adbba7be39a64230bab160ce77ce7586`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:57:39.982494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a8fcaa63-08df-4d0d-a550-0dc169d6e64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:57:40.052418 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1113/2246] 10.222.0.36 () {58 vars in 1018 bytes} [Sun Oct 30 00:57:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:57:40.064881 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c5734-e203-4755-bce5-4f838b6e1dfd None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:57:40.066628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c10c5734-e203-4755-bce5-4f838b6e1dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:57:40.125337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c5734-e203-4755-bce5-4f838b6e1dfd None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:57:40.128846 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8eb15f-8ee8-4708-8330-3166631b9295 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cf9451f2ef7478589b4a4278cb03252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:57:40.129444 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eb15f-8ee8-4708-8330-3166631b9295 service glance] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:40.129720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eb15f-8ee8-4708-8330-3166631b9295 service glance] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:40.129995 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8eb15f-8ee8-4708-8330-3166631b9295 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:40.140389 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8eb15f-8ee8-4708-8330-3166631b9295 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:57:40.141810 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8eb15f-8ee8-4708-8330-3166631b9295 service glance] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:57:40.152999 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1134/2247] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:57:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:57:40.224865 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0cebfa-7eae-4b0b-8fa1-5342be474d73 None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:40.225231 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0cebfa-7eae-4b0b-8fa1-5342be474d73 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:40.225502 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0cebfa-7eae-4b0b-8fa1-5342be474d73 None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:40.226885 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1114/2248] 10.222.0.36 () {54 vars in 921 bytes} [Sun Oct 30 00:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:57:40.231940 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bf9f6-e3a5-4ef1-931e-efea14d4ddc0 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:40.232193 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bf9f6-e3a5-4ef1-931e-efea14d4ddc0 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:40.232461 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bf9f6-e3a5-4ef1-931e-efea14d4ddc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:40.252100 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-5f1bf9f6-e3a5-4ef1-931e-efea14d4ddc0 None None] MFA Rules not processed for user `fb39b6cc97a1456bb1312f49118c6e10`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:57:40.253368 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-5f1bf9f6-e3a5-4ef1-931e-efea14d4ddc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:57:40.308348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1135/2249] 10.222.0.36 () {58 vars in 1018 bytes} [Sun Oct 30 00:57:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:57:40.319621 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0cebfa-7eae-4b0b-8fa1-5342be474d73 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:57:40.321483 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-7e0cebfa-7eae-4b0b-8fa1-5342be474d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:57:40.339506 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0cebfa-7eae-4b0b-8fa1-5342be474d73 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:57:40.343779 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b7650-d989-436d-9c3d-a6477d2087af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:57:40.344424 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b7650-d989-436d-9c3d-a6477d2087af service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:40.344706 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b7650-d989-436d-9c3d-a6477d2087af service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:40.344979 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b7650-d989-436d-9c3d-a6477d2087af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:40.352501 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b7650-d989-436d-9c3d-a6477d2087af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:57:40.353933 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b7650-d989-436d-9c3d-a6477d2087af service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:57:40.362221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1115/2250] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:57:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:57:40.446261 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ec3f3-2d2f-4a9e-a547-76fa17739f02 None None] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:40.446595 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ec3f3-2d2f-4a9e-a547-76fa17739f02 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:40.446849 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ec3f3-2d2f-4a9e-a547-76fa17739f02 None None] PATH_INFO: `/` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:40.449674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1136/2251] 10.222.0.36 () {54 vars in 921 bytes} [Sun Oct 30 00:57:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 30 00:57:40.455041 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e7c64-7313-4079-ad07-974ea6dbde21 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:40.455400 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e7c64-7313-4079-ad07-974ea6dbde21 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:40.455711 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e7c64-7313-4079-ad07-974ea6dbde21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:40.477514 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-299e7c64-7313-4079-ad07-974ea6dbde21 None None] MFA Rules not processed for user `adbba7be39a64230bab160ce77ce7586`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:57:40.478716 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-299e7c64-7313-4079-ad07-974ea6dbde21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:57:40.487991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1116/2252] 10.222.0.36 () {58 vars in 1018 bytes} [Sun Oct 30 00:57:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:57:40.498644 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ec3f3-2d2f-4a9e-a547-76fa17739f02 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:57:40.512633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ec3f3-2d2f-4a9e-a547-76fa17739f02 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:57:40.516321 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339002c1-5393-4379-93ca-52164a877864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:57:40.516916 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339002c1-5393-4379-93ca-52164a877864 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:40.517195 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339002c1-5393-4379-93ca-52164a877864 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:40.517467 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339002c1-5393-4379-93ca-52164a877864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:40.528190 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339002c1-5393-4379-93ca-52164a877864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:57:40.529623 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339002c1-5393-4379-93ca-52164a877864 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:57:40.540230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1137/2253] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:57:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:57:42.310880 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281856d5-1bf2-4a37-9022-89646512812b None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:42.311406 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281856d5-1bf2-4a37-9022-89646512812b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:42.311516 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281856d5-1bf2-4a37-9022-89646512812b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:42.331724 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-281856d5-1bf2-4a37-9022-89646512812b None None] MFA Rules not processed for user `8a91d3b9c4834d729211ec367e235889`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:57:42.332977 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-281856d5-1bf2-4a37-9022-89646512812b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:57:42.340397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1117/2254] 10.222.0.36 () {58 vars in 1009 bytes} [Sun Oct 30 00:57:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:57:42.352910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339002c1-5393-4379-93ca-52164a877864 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:57:42.364578 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339002c1-5393-4379-93ca-52164a877864 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:57:42.369500 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204e68c-da83-433b-95d2-3c3467da3877 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f4c779b8422432fb06cdf996841b9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:57:42.370593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204e68c-da83-433b-95d2-3c3467da3877 service swift] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:57:42.370871 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204e68c-da83-433b-95d2-3c3467da3877 service swift] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:57:42.371141 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204e68c-da83-433b-95d2-3c3467da3877 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:57:42.379414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7204e68c-da83-433b-95d2-3c3467da3877 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:57:42.381018 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7204e68c-da83-433b-95d2-3c3467da3877 service swift] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:57:42.390100 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1138/2255] 10.222.0.36 () {60 vars in 1384 bytes} [Sun Oct 30 00:57:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 00:57:52.326431 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=43 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:57:52.385817 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=61 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:02.647999 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe0cbb-cf29-44a5-b173-2ebe6ef9f92c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:02.648477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe0cbb-cf29-44a5-b173-2ebe6ef9f92c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:02.648577 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe0cbb-cf29-44a5-b173-2ebe6ef9f92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:02.670041 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-acfe0cbb-cf29-44a5-b173-2ebe6ef9f92c None None] MFA Rules not processed for user `8a91d3b9c4834d729211ec367e235889`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:58:02.671259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-acfe0cbb-cf29-44a5-b173-2ebe6ef9f92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:58:02.678602 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1118/2256] 10.222.0.36 () {58 vars in 967 bytes} [Sun Oct 30 00:58:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:58:02.690955 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204e68c-da83-433b-95d2-3c3467da3877 service swift] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:58:02.701538 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204e68c-da83-433b-95d2-3c3467da3877 service swift] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:58:02.706650 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2c51b-82e6-4fe3-8b55-2db88e15f97e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f4c779b8422432fb06cdf996841b9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:58:02.707324 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2c51b-82e6-4fe3-8b55-2db88e15f97e service swift] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:02.707674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2c51b-82e6-4fe3-8b55-2db88e15f97e service swift] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:02.708012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2c51b-82e6-4fe3-8b55-2db88e15f97e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:02.717370 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2c51b-82e6-4fe3-8b55-2db88e15f97e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:58:02.719077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2c51b-82e6-4fe3-8b55-2db88e15f97e service swift] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:58:02.727611 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1139/2257] 10.222.0.36 () {60 vars in 1384 bytes} [Sun Oct 30 00:58:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 00:58:12.664280 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:12.724745 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=6 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:25.846838 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe0cbb-cf29-44a5-b173-2ebe6ef9f92c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:58:25.848280 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-acfe0cbb-cf29-44a5-b173-2ebe6ef9f92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:58:25.911597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe0cbb-cf29-44a5-b173-2ebe6ef9f92c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:58:25.915205 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6645a09-c5c1-4260-a26c-832a847b8f77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:58:25.915828 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6645a09-c5c1-4260-a26c-832a847b8f77 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:25.916134 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6645a09-c5c1-4260-a26c-832a847b8f77 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:25.916413 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6645a09-c5c1-4260-a26c-832a847b8f77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:25.918150 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d6645a09-c5c1-4260-a26c-832a847b8f77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:58:25.933220 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6645a09-c5c1-4260-a26c-832a847b8f77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:58:25.934660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6645a09-c5c1-4260-a26c-832a847b8f77 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:58:25.946558 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1119/2258] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:58:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:58:29.365308 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2c51b-82e6-4fe3-8b55-2db88e15f97e service swift] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:58:29.366726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4ba2c51b-82e6-4fe3-8b55-2db88e15f97e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:58:29.379126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2c51b-82e6-4fe3-8b55-2db88e15f97e service swift] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:58:29.382648 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128476d-e6b3-46f5-b012-aced30bfde79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:58:29.383229 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128476d-e6b3-46f5-b012-aced30bfde79 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:29.383504 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128476d-e6b3-46f5-b012-aced30bfde79 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:29.383774 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128476d-e6b3-46f5-b012-aced30bfde79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:29.385446 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-9128476d-e6b3-46f5-b012-aced30bfde79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:58:29.465188 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128476d-e6b3-46f5-b012-aced30bfde79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:58:29.466608 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128476d-e6b3-46f5-b012-aced30bfde79 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:58:29.494088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1140/2259] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:58:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:58:35.469518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6645a09-c5c1-4260-a26c-832a847b8f77 service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:58:35.480335 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6645a09-c5c1-4260-a26c-832a847b8f77 service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:58:35.483947 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2c089-85eb-44c5-9bdc-6f56d57a3210 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:58:35.484809 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2c089-85eb-44c5-9bdc-6f56d57a3210 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:35.485455 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2c089-85eb-44c5-9bdc-6f56d57a3210 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:35.485455 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2c089-85eb-44c5-9bdc-6f56d57a3210 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:35.487158 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-6ca2c089-85eb-44c5-9bdc-6f56d57a3210 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:58:35.500271 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2c089-85eb-44c5-9bdc-6f56d57a3210 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:58:35.501712 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2c089-85eb-44c5-9bdc-6f56d57a3210 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:58:35.512821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1120/2260] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:58:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:58:39.491445 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=41 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:45.507751 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=32 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:55.458508 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128476d-e6b3-46f5-b012-aced30bfde79 service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:58:55.459926 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-9128476d-e6b3-46f5-b012-aced30bfde79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:58:55.471851 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128476d-e6b3-46f5-b012-aced30bfde79 service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:58:55.476233 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe079b-16a6-4d68-ba99-40ffde3c3f68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:58:55.476833 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe079b-16a6-4d68-ba99-40ffde3c3f68 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:55.477108 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe079b-16a6-4d68-ba99-40ffde3c3f68 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:55.477381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe079b-16a6-4d68-ba99-40ffde3c3f68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:55.479029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-3dfe079b-16a6-4d68-ba99-40ffde3c3f68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:58:55.544133 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe079b-16a6-4d68-ba99-40ffde3c3f68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:58:55.545568 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe079b-16a6-4d68-ba99-40ffde3c3f68 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:58:55.556597 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1141/2261] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:58:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:58:58.663530 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fbd01-3973-435a-be30-ca1b22fc512e None None] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:58.664011 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fbd01-3973-435a-be30-ca1b22fc512e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:58.664231 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fbd01-3973-435a-be30-ca1b22fc512e None None] PATH_INFO: `/` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:58.666079 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1121/2262] 10.222.0.36 () {54 vars in 961 bytes} [Sun Oct 30 00:58:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 30 00:58:58.671309 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d1cb8-2edb-49ac-bdcb-849c47f78b2d None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:58.671803 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d1cb8-2edb-49ac-bdcb-849c47f78b2d None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:58.671917 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d1cb8-2edb-49ac-bdcb-849c47f78b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:58.712963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-f90d1cb8-2edb-49ac-bdcb-849c47f78b2d None None] MFA Rules not processed for user `5ded89d48d6c4a15a5c5829f594f8776`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:58:58.714185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f90d1cb8-2edb-49ac-bdcb-849c47f78b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:58:58.733796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fbd01-3973-435a-be30-ca1b22fc512e None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:58:58.736350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-9d8fbd01-3973-435a-be30-ca1b22fc512e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:58:58.755315 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fbd01-3973-435a-be30-ca1b22fc512e None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:58:58.761520 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efbd62-5688-4a0a-8027-68876b12c2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:58:58.762163 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efbd62-5688-4a0a-8027-68876b12c2ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:58.762448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efbd62-5688-4a0a-8027-68876b12c2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:58.762744 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efbd62-5688-4a0a-8027-68876b12c2ea admin admin] PATH_INFO: `/v3/users/5ac3fdb86ac24fa493706f84e1aa40a8` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:58.763900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efbd62-5688-4a0a-8027-68876b12c2ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ac3fdb86ac24fa493706f84e1aa40a8)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:58:58.766190 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efbd62-5688-4a0a-8027-68876b12c2ea admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:58:58.787914 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1142/2263] 10.222.0.36 () {58 vars in 1058 bytes} [Sun Oct 30 00:58:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:58:58.795792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d1cb8-2edb-49ac-bdcb-849c47f78b2d None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:58:58.806627 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d1cb8-2edb-49ac-bdcb-849c47f78b2d None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:58:58.811871 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205fed6e-5531-4876-8285-04129eb3f133 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:58:58.812498 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fed6e-5531-4876-8285-04129eb3f133 service placement] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:58.812774 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fed6e-5531-4876-8285-04129eb3f133 service placement] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:58.813046 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fed6e-5531-4876-8285-04129eb3f133 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:58.814737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-205fed6e-5531-4876-8285-04129eb3f133 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:58:58.817707 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a1efbd62-5688-4a0a-8027-68876b12c2ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ac3fdb86ac24fa493706f84e1aa40a8'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:58:58.818458 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a1efbd62-5688-4a0a-8027-68876b12c2ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ac3fdb86ac24fa493706f84e1aa40a8'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:58:58.827432 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205fed6e-5531-4876-8285-04129eb3f133 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:58:58.829091 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205fed6e-5531-4876-8285-04129eb3f133 service placement] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:58:58.830809 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a1efbd62-5688-4a0a-8027-68876b12c2ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ac3fdb86ac24fa493706f84e1aa40a8'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:58:58.838541 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a1efbd62-5688-4a0a-8027-68876b12c2ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ac3fdb86ac24fa493706f84e1aa40a8'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:58:58.839229 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-a1efbd62-5688-4a0a-8027-68876b12c2ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ac3fdb86ac24fa493706f84e1aa40a8'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:58:58.839952 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1143/2264] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:58:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:58:58.855935 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1122/2265] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:58:58 2022] DELETE /identity/v3/users/5ac3fdb86ac24fa493706f84e1aa40a8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:58:59.027227 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205fed6e-5531-4876-8285-04129eb3f133 service placement] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:58:59.038289 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205fed6e-5531-4876-8285-04129eb3f133 service placement] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:58:59.100209 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99755d8-b521-4202-8008-5200259783af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:58:59.100843 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99755d8-b521-4202-8008-5200259783af admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:59.101123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99755d8-b521-4202-8008-5200259783af admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:59.101398 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99755d8-b521-4202-8008-5200259783af admin admin] PATH_INFO: `/v3/projects/bb5d551adcf64bb8aea88a6c142cea5e` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:59.102517 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99755d8-b521-4202-8008-5200259783af admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb5d551adcf64bb8aea88a6c142cea5e)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:58:59.104468 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99755d8-b521-4202-8008-5200259783af admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:58:59.110600 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:59.154128 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f99755d8-b521-4202-8008-5200259783af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb5d551adcf64bb8aea88a6c142cea5e', 'request_id': 'req-f99755d8-b521-4202-8008-5200259783af'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:58:59.164306 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f99755d8-b521-4202-8008-5200259783af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb5d551adcf64bb8aea88a6c142cea5e', 'request_id': 'req-f99755d8-b521-4202-8008-5200259783af'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:58:59.165184 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efbd62-5688-4a0a-8027-68876b12c2ea admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:58:59.166711 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-a1efbd62-5688-4a0a-8027-68876b12c2ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:58:59.171442 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f99755d8-b521-4202-8008-5200259783af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb5d551adcf64bb8aea88a6c142cea5e', 'request_id': 'req-f99755d8-b521-4202-8008-5200259783af'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:58:59.183565 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f99755d8-b521-4202-8008-5200259783af admin admin] The token cache is being invalidate because project bb5d551adcf64bb8aea88a6c142cea5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:58:59.183853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f99755d8-b521-4202-8008-5200259783af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:58:59.185348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f99755d8-b521-4202-8008-5200259783af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:58:59.190778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1144/2266] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:58:59 2022] DELETE /identity/v3/projects/bb5d551adcf64bb8aea88a6c142cea5e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:58:59.192747 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a7c5c-3259-41d8-b267-e85a068f0530 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:59.193315 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a7c5c-3259-41d8-b267-e85a068f0530 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:59.193646 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a7c5c-3259-41d8-b267-e85a068f0530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:59.215867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-620a7c5c-3259-41d8-b267-e85a068f0530 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:58:59.217129 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-620a7c5c-3259-41d8-b267-e85a068f0530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:58:59.233924 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efbd62-5688-4a0a-8027-68876b12c2ea admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:58:59.283433 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1145/2267] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:58:59.291588 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94dfb2e-f0c6-4e58-8539-85e28b08b3f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:58:59.292238 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dfb2e-f0c6-4e58-8539-85e28b08b3f7 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:59.292381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a3f53-ad4b-4584-b418-5bf7c6457403 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:59.292578 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a3f53-ad4b-4584-b418-5bf7c6457403 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:59.292690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dfb2e-f0c6-4e58-8539-85e28b08b3f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:59.292857 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a3f53-ad4b-4584-b418-5bf7c6457403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:59.292955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dfb2e-f0c6-4e58-8539-85e28b08b3f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:59.294789 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c94dfb2e-f0c6-4e58-8539-85e28b08b3f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:58:59.311968 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-3e6a3f53-ad4b-4584-b418-5bf7c6457403 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:58:59.313197 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-3e6a3f53-ad4b-4584-b418-5bf7c6457403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:58:59.321383 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1146/2268] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:58:59.325637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a3f53-ad4b-4584-b418-5bf7c6457403 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:58:59.335116 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a3f53-ad4b-4584-b418-5bf7c6457403 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:58:59.339353 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:59.339668 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5f533-7c98-4969-a58d-099a2b554c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:58:59.340325 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5f533-7c98-4969-a58d-099a2b554c79 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:59.340855 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5f533-7c98-4969-a58d-099a2b554c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:59.341160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5f533-7c98-4969-a58d-099a2b554c79 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:59.341946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5f533-7c98-4969-a58d-099a2b554c79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:58:59.342246 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5f533-7c98-4969-a58d-099a2b554c79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:58:59.343892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5f533-7c98-4969-a58d-099a2b554c79 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:58:59.357726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1147/2269] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 00:58:59.361930 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5f533-7c98-4969-a58d-099a2b554c79 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:58:59.372533 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5f533-7c98-4969-a58d-099a2b554c79 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:58:59.377246 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f849f7-bf1a-4152-89cc-67c2905385e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:58:59.377943 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f849f7-bf1a-4152-89cc-67c2905385e5 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:59.378220 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f849f7-bf1a-4152-89cc-67c2905385e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:59.378491 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f849f7-bf1a-4152-89cc-67c2905385e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:59.379109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f849f7-bf1a-4152-89cc-67c2905385e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:58:59.380366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94dfb2e-f0c6-4e58-8539-85e28b08b3f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:58:59.381536 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f849f7-bf1a-4152-89cc-67c2905385e5 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:58:59.381808 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94dfb2e-f0c6-4e58-8539-85e28b08b3f7 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:58:59.401950 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1123/2270] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:58:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:58:59.419497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1148/2271] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:58:59 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:58:59.423756 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94dfb2e-f0c6-4e58-8539-85e28b08b3f7 service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:58:59.434288 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94dfb2e-f0c6-4e58-8539-85e28b08b3f7 service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:58:59.493761 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176093c3-86fa-4004-9e18-ab7aac485a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:58:59.494368 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176093c3-86fa-4004-9e18-ab7aac485a5e admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:59.494651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176093c3-86fa-4004-9e18-ab7aac485a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:59.494919 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176093c3-86fa-4004-9e18-ab7aac485a5e admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:59.495542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176093c3-86fa-4004-9e18-ab7aac485a5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:58:59.497484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176093c3-86fa-4004-9e18-ab7aac485a5e admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:58:59.524158 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1124/2272] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:58:59 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:58:59.528233 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f849f7-bf1a-4152-89cc-67c2905385e5 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:58:59.539321 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f849f7-bf1a-4152-89cc-67c2905385e5 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:58:59.543900 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba89e8-98b2-4502-80f0-485c7a10fe7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:58:59.544530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba89e8-98b2-4502-80f0-485c7a10fe7f admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:59.544927 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba89e8-98b2-4502-80f0-485c7a10fe7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:59.545208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba89e8-98b2-4502-80f0-485c7a10fe7f admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:59.545843 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba89e8-98b2-4502-80f0-485c7a10fe7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:58:59.547410 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba89e8-98b2-4502-80f0-485c7a10fe7f admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:58:59.558826 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1149/2273] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:58:59.562621 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176093c3-86fa-4004-9e18-ab7aac485a5e admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:58:59.574501 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176093c3-86fa-4004-9e18-ab7aac485a5e admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:58:59.579710 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e84bd8-a5cc-4136-8bb0-8ec74d97bc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:58:59.580347 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e84bd8-a5cc-4136-8bb0-8ec74d97bc80 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:59.580629 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e84bd8-a5cc-4136-8bb0-8ec74d97bc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:59.580907 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e84bd8-a5cc-4136-8bb0-8ec74d97bc80 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:59.581551 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e84bd8-a5cc-4136-8bb0-8ec74d97bc80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:58:59.583832 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e84bd8-a5cc-4136-8bb0-8ec74d97bc80 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:58:59.594548 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1125/2274] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:58:59.599048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba89e8-98b2-4502-80f0-485c7a10fe7f admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:58:59.608974 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba89e8-98b2-4502-80f0-485c7a10fe7f admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:58:59.614992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344a95f-dbf4-4182-9b4b-aa8bdd0db0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:58:59.615643 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344a95f-dbf4-4182-9b4b-aa8bdd0db0a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:59.615919 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344a95f-dbf4-4182-9b4b-aa8bdd0db0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:59.616220 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344a95f-dbf4-4182-9b4b-aa8bdd0db0a9 admin admin] PATH_INFO: `/v3/projects/dc70ef555fc84a51b3521007d00b77e3/users/7353722ccb7548a6b92c89d4917fefaa/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:59.633067 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344a95f-dbf4-4182-9b4b-aa8bdd0db0a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc70ef555fc84a51b3521007d00b77e3, user_id=7353722ccb7548a6b92c89d4917fefaa, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:58:59.635828 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344a95f-dbf4-4182-9b4b-aa8bdd0db0a9 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:58:59.650873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1150/2275] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:58:59 2022] PUT /identity/v3/projects/dc70ef555fc84a51b3521007d00b77e3/users/7353722ccb7548a6b92c89d4917fefaa/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:58:59.656587 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8b168-fe9c-409f-81ad-ee11b9d61403 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:59.656908 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8b168-fe9c-409f-81ad-ee11b9d61403 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:59.657242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8b168-fe9c-409f-81ad-ee11b9d61403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:59.678701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-d9d8b168-fe9c-409f-81ad-ee11b9d61403 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:58:59.679920 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d9d8b168-fe9c-409f-81ad-ee11b9d61403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:58:59.746497 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1126/2276] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:58:59.756825 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344a95f-dbf4-4182-9b4b-aa8bdd0db0a9 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:58:59.758558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a344a95f-dbf4-4182-9b4b-aa8bdd0db0a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:58:59.777593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:58:59.827887 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344a95f-dbf4-4182-9b4b-aa8bdd0db0a9 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:58:59.832143 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00022eb-773f-40ac-ac50-b8b907235344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:58:59.833323 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00022eb-773f-40ac-ac50-b8b907235344 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:58:59.833604 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00022eb-773f-40ac-ac50-b8b907235344 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:58:59.833971 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00022eb-773f-40ac-ac50-b8b907235344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:58:59.843639 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00022eb-773f-40ac-ac50-b8b907235344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:58:59.845115 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00022eb-773f-40ac-ac50-b8b907235344 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:58:59.855637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1151/2277] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:58:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:59:04.374390 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2d820-452f-4ba3-a23f-d3b3e1e04c31 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:04.374878 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2d820-452f-4ba3-a23f-d3b3e1e04c31 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:04.375081 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2d820-452f-4ba3-a23f-d3b3e1e04c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:04.418638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-4fc2d820-452f-4ba3-a23f-d3b3e1e04c31 None None] MFA Rules not processed for user `7353722ccb7548a6b92c89d4917fefaa`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:04.420711 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4fc2d820-452f-4ba3-a23f-d3b3e1e04c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:04.481494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:04.545502 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1127/2278] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:04.560518 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00022eb-773f-40ac-ac50-b8b907235344 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:04.562088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e00022eb-773f-40ac-ac50-b8b907235344 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:04.656368 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00022eb-773f-40ac-ac50-b8b907235344 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:04.663740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3403bca-d47c-41ae-b741-94a805666943 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:04.664422 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3403bca-d47c-41ae-b741-94a805666943 service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:04.664709 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3403bca-d47c-41ae-b741-94a805666943 service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:04.664995 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3403bca-d47c-41ae-b741-94a805666943 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:04.674235 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3403bca-d47c-41ae-b741-94a805666943 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:04.675810 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3403bca-d47c-41ae-b741-94a805666943 service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:04.688366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1152/2279] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:59:06.321648 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7874ba-6927-4c97-9efe-20343f25e370 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:06.322134 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7874ba-6927-4c97-9efe-20343f25e370 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:06.322230 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7874ba-6927-4c97-9efe-20343f25e370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:06.342942 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-bd7874ba-6927-4c97-9efe-20343f25e370 None None] MFA Rules not processed for user `7353722ccb7548a6b92c89d4917fefaa`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:06.344226 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-bd7874ba-6927-4c97-9efe-20343f25e370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:06.352838 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1128/2280] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:06.717028 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3403bca-d47c-41ae-b741-94a805666943 service nova] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:06.721052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f3403bca-d47c-41ae-b741-94a805666943 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:06.736629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3403bca-d47c-41ae-b741-94a805666943 service nova] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:06.740698 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162ac2e-b6b7-45c7-bead-d1f1c1e937dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:06.741359 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162ac2e-b6b7-45c7-bead-d1f1c1e937dc service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:06.741647 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162ac2e-b6b7-45c7-bead-d1f1c1e937dc service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:06.741923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162ac2e-b6b7-45c7-bead-d1f1c1e937dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:06.743829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-0162ac2e-b6b7-45c7-bead-d1f1c1e937dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:06.755381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162ac2e-b6b7-45c7-bead-d1f1c1e937dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:06.757120 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162ac2e-b6b7-45c7-bead-d1f1c1e937dc service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:06.767882 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1153/2281] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:59:06.856464 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7874ba-6927-4c97-9efe-20343f25e370 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:06.867725 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7874ba-6927-4c97-9efe-20343f25e370 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:06.877805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf618a-9546-4bf3-8f34-783cf8c54078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:06.878448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf618a-9546-4bf3-8f34-783cf8c54078 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:06.878842 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf618a-9546-4bf3-8f34-783cf8c54078 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:06.879699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf618a-9546-4bf3-8f34-783cf8c54078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:06.881687 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c5cf618a-9546-4bf3-8f34-783cf8c54078 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:06.898692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf618a-9546-4bf3-8f34-783cf8c54078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:06.900500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf618a-9546-4bf3-8f34-783cf8c54078 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:06.913555 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1129/2282] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:59:07.190109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162ac2e-b6b7-45c7-bead-d1f1c1e937dc service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:07.191542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-0162ac2e-b6b7-45c7-bead-d1f1c1e937dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:07.323460 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162ac2e-b6b7-45c7-bead-d1f1c1e937dc service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:07.330371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62635859-c41b-4453-8571-06b516548993 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:07.331100 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62635859-c41b-4453-8571-06b516548993 service placement] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:07.331440 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62635859-c41b-4453-8571-06b516548993 service placement] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:07.331789 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62635859-c41b-4453-8571-06b516548993 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:07.333953 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-62635859-c41b-4453-8571-06b516548993 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:07.349704 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62635859-c41b-4453-8571-06b516548993 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:07.352022 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62635859-c41b-4453-8571-06b516548993 service placement] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:07.368185 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1154/2283] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:59:10.227531 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf618a-9546-4bf3-8f34-783cf8c54078 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:10.238272 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf618a-9546-4bf3-8f34-783cf8c54078 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:10.244355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1697d54-8a2d-4e17-848e-784bc0bdc203 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:10.245620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1697d54-8a2d-4e17-848e-784bc0bdc203 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:10.245620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1697d54-8a2d-4e17-848e-784bc0bdc203 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:10.245620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1697d54-8a2d-4e17-848e-784bc0bdc203 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:10.247815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d1697d54-8a2d-4e17-848e-784bc0bdc203 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:10.267074 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1697d54-8a2d-4e17-848e-784bc0bdc203 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:10.268781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1697d54-8a2d-4e17-848e-784bc0bdc203 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:10.284448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1130/2284] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:59:10.942925 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62635859-c41b-4453-8571-06b516548993 service placement] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:10.954199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62635859-c41b-4453-8571-06b516548993 service placement] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:10.961448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd400cc-c726-4da0-8b60-a2af1dd8a388 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:10.962070 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd400cc-c726-4da0-8b60-a2af1dd8a388 service placement] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:10.962734 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd400cc-c726-4da0-8b60-a2af1dd8a388 service placement] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:10.963182 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd400cc-c726-4da0-8b60-a2af1dd8a388 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:10.965042 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-fcd400cc-c726-4da0-8b60-a2af1dd8a388 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:10.979547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd400cc-c726-4da0-8b60-a2af1dd8a388 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:10.981569 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd400cc-c726-4da0-8b60-a2af1dd8a388 service placement] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:10.989044 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:10.994635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1155/2285] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:59:11.036978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1697d54-8a2d-4e17-848e-784bc0bdc203 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:11.039083 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d1697d54-8a2d-4e17-848e-784bc0bdc203 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:11.058486 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1697d54-8a2d-4e17-848e-784bc0bdc203 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:11.063990 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919b0eb-8696-4f8e-8ec1-321865402225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:11.064825 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919b0eb-8696-4f8e-8ec1-321865402225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:11.065206 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919b0eb-8696-4f8e-8ec1-321865402225 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:11.065844 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919b0eb-8696-4f8e-8ec1-321865402225 admin admin] PATH_INFO: `/v3/users/df1ee15eb3bc468188a9d8c1af46529c` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:11.067056 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919b0eb-8696-4f8e-8ec1-321865402225 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df1ee15eb3bc468188a9d8c1af46529c)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:11.069788 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919b0eb-8696-4f8e-8ec1-321865402225 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:11.130520 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b919b0eb-8696-4f8e-8ec1-321865402225 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df1ee15eb3bc468188a9d8c1af46529c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:11.131197 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b919b0eb-8696-4f8e-8ec1-321865402225 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df1ee15eb3bc468188a9d8c1af46529c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:11.147889 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b919b0eb-8696-4f8e-8ec1-321865402225 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df1ee15eb3bc468188a9d8c1af46529c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:11.155584 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b919b0eb-8696-4f8e-8ec1-321865402225 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df1ee15eb3bc468188a9d8c1af46529c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:11.156345 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b919b0eb-8696-4f8e-8ec1-321865402225 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df1ee15eb3bc468188a9d8c1af46529c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:11.174361 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1131/2286] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:59:11 2022] DELETE /identity/v3/users/df1ee15eb3bc468188a9d8c1af46529c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:59:11.367333 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd400cc-c726-4da0-8b60-a2af1dd8a388 service placement] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:11.389076 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd400cc-c726-4da0-8b60-a2af1dd8a388 service placement] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:11.455664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7795772-d7b1-4e70-8200-79d0fb3b945d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:11.456417 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7795772-d7b1-4e70-8200-79d0fb3b945d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:11.456795 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7795772-d7b1-4e70-8200-79d0fb3b945d admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:11.457214 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7795772-d7b1-4e70-8200-79d0fb3b945d admin admin] PATH_INFO: `/v3/projects/0f3b2aeb0e094f539e4affd66ea66272` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:11.458350 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7795772-d7b1-4e70-8200-79d0fb3b945d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f3b2aeb0e094f539e4affd66ea66272)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:11.460394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7795772-d7b1-4e70-8200-79d0fb3b945d admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:11.510114 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-e7795772-d7b1-4e70-8200-79d0fb3b945d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f3b2aeb0e094f539e4affd66ea66272', 'request_id': 'req-e7795772-d7b1-4e70-8200-79d0fb3b945d'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:11.517814 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-e7795772-d7b1-4e70-8200-79d0fb3b945d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f3b2aeb0e094f539e4affd66ea66272', 'request_id': 'req-e7795772-d7b1-4e70-8200-79d0fb3b945d'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:11.524089 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-e7795772-d7b1-4e70-8200-79d0fb3b945d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f3b2aeb0e094f539e4affd66ea66272', 'request_id': 'req-e7795772-d7b1-4e70-8200-79d0fb3b945d'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:11.535113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-e7795772-d7b1-4e70-8200-79d0fb3b945d admin admin] The token cache is being invalidate because project 0f3b2aeb0e094f539e4affd66ea66272 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:59:11.535340 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-e7795772-d7b1-4e70-8200-79d0fb3b945d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:11.535977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-e7795772-d7b1-4e70-8200-79d0fb3b945d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:11.538143 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1156/2287] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:59:11 2022] DELETE /identity/v3/projects/0f3b2aeb0e094f539e4affd66ea66272 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:11.543144 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fa256-a087-4f6f-8f72-ea5a331aff0a None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:11.543671 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fa256-a087-4f6f-8f72-ea5a331aff0a None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:11.543774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fa256-a087-4f6f-8f72-ea5a331aff0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:11.564339 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-e56fa256-a087-4f6f-8f72-ea5a331aff0a None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:11.565562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e56fa256-a087-4f6f-8f72-ea5a331aff0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:11.624985 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1132/2288] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:11.633226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd97c71-6438-4fc5-a9e4-e6970e39fd31 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:11.633507 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd97c71-6438-4fc5-a9e4-e6970e39fd31 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:11.633776 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd97c71-6438-4fc5-a9e4-e6970e39fd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:11.654151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-4fd97c71-6438-4fc5-a9e4-e6970e39fd31 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:11.655346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-4fd97c71-6438-4fc5-a9e4-e6970e39fd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:11.663806 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1157/2289] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:11.668229 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fa256-a087-4f6f-8f72-ea5a331aff0a None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:11.673019 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:11.680341 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fa256-a087-4f6f-8f72-ea5a331aff0a None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:11.685883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71daeac4-7823-4444-95d0-e895390f9d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:11.686521 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71daeac4-7823-4444-95d0-e895390f9d78 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:11.686804 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71daeac4-7823-4444-95d0-e895390f9d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:11.687079 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71daeac4-7823-4444-95d0-e895390f9d78 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:11.687758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71daeac4-7823-4444-95d0-e895390f9d78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:59:11.688065 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71daeac4-7823-4444-95d0-e895390f9d78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:11.689762 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71daeac4-7823-4444-95d0-e895390f9d78 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:11.702207 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1133/2290] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:59:11.706658 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd97c71-6438-4fc5-a9e4-e6970e39fd31 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:11.716908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd97c71-6438-4fc5-a9e4-e6970e39fd31 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:11.721532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d4f98-5d30-464a-9e43-041bb02401b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:11.722126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d4f98-5d30-464a-9e43-041bb02401b2 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:11.722403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d4f98-5d30-464a-9e43-041bb02401b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:11.722674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d4f98-5d30-464a-9e43-041bb02401b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:11.723310 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d4f98-5d30-464a-9e43-041bb02401b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:11.725371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d4f98-5d30-464a-9e43-041bb02401b2 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:11.752158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1158/2291] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:59:11 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:59:11.757177 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71daeac4-7823-4444-95d0-e895390f9d78 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:11.767198 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71daeac4-7823-4444-95d0-e895390f9d78 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:11.824690 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff7be4-4e8b-4470-a48f-2ec5edc3fe29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:11.825187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff7be4-4e8b-4470-a48f-2ec5edc3fe29 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:11.825465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff7be4-4e8b-4470-a48f-2ec5edc3fe29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:11.825737 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff7be4-4e8b-4470-a48f-2ec5edc3fe29 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:11.826358 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff7be4-4e8b-4470-a48f-2ec5edc3fe29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:11.828298 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff7be4-4e8b-4470-a48f-2ec5edc3fe29 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:11.852263 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1134/2292] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:59:11 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:59:11.856343 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d4f98-5d30-464a-9e43-041bb02401b2 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:11.867632 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d4f98-5d30-464a-9e43-041bb02401b2 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:11.872460 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43633da6-a2b5-4e80-a2e9-b1a322585218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:11.873077 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43633da6-a2b5-4e80-a2e9-b1a322585218 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:11.873357 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43633da6-a2b5-4e80-a2e9-b1a322585218 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:11.873636 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43633da6-a2b5-4e80-a2e9-b1a322585218 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:11.874283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43633da6-a2b5-4e80-a2e9-b1a322585218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:11.875878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43633da6-a2b5-4e80-a2e9-b1a322585218 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:11.887920 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1159/2293] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:59:11.891798 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff7be4-4e8b-4470-a48f-2ec5edc3fe29 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:11.905821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff7be4-4e8b-4470-a48f-2ec5edc3fe29 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:11.910808 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91964726-0773-4035-8a61-ba672370e289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:11.911787 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91964726-0773-4035-8a61-ba672370e289 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:11.912297 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91964726-0773-4035-8a61-ba672370e289 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:11.912562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91964726-0773-4035-8a61-ba672370e289 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:11.913310 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91964726-0773-4035-8a61-ba672370e289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:11.915018 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91964726-0773-4035-8a61-ba672370e289 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:11.926108 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1135/2294] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:59:11.931428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43633da6-a2b5-4e80-a2e9-b1a322585218 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:11.941363 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43633da6-a2b5-4e80-a2e9-b1a322585218 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:11.947338 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa699a9-a34d-47d5-bd6d-afec12ce234c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:11.948069 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa699a9-a34d-47d5-bd6d-afec12ce234c admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:11.948448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa699a9-a34d-47d5-bd6d-afec12ce234c admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:11.948807 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa699a9-a34d-47d5-bd6d-afec12ce234c admin admin] PATH_INFO: `/v3/projects/3570a646af4c45398376695d79e1b4c3/users/1ca6e5391e2348f9b8985b383742972e/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:11.964238 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa699a9-a34d-47d5-bd6d-afec12ce234c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3570a646af4c45398376695d79e1b4c3, user_id=1ca6e5391e2348f9b8985b383742972e, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:11.967240 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa699a9-a34d-47d5-bd6d-afec12ce234c admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:11.981669 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1160/2295] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:59:11 2022] PUT /identity/v3/projects/3570a646af4c45398376695d79e1b4c3/users/1ca6e5391e2348f9b8985b383742972e/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:59:11.994231 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa334447-08a6-48fe-a209-35b6c05443cb None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:11.994543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa334447-08a6-48fe-a209-35b6c05443cb None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:11.994823 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa334447-08a6-48fe-a209-35b6c05443cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:12.024725 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-fa334447-08a6-48fe-a209-35b6c05443cb None None] MFA Rules not processed for user `1ca6e5391e2348f9b8985b383742972e`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:12.025999 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-fa334447-08a6-48fe-a209-35b6c05443cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:12.089975 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1136/2296] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:12.094784 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa699a9-a34d-47d5-bd6d-afec12ce234c admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:12.125484 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:12.160998 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa699a9-a34d-47d5-bd6d-afec12ce234c admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:12.166483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d585d2-1e7e-4482-b044-da81a8ab55b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:12.167085 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d585d2-1e7e-4482-b044-da81a8ab55b4 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:12.167364 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d585d2-1e7e-4482-b044-da81a8ab55b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:12.167640 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d585d2-1e7e-4482-b044-da81a8ab55b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:12.168289 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d585d2-1e7e-4482-b044-da81a8ab55b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:12.170193 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d585d2-1e7e-4482-b044-da81a8ab55b4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:12.196199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1161/2297] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:59:12 2022] POST /identity/v3/projects => generated 361 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:59:12.200754 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa334447-08a6-48fe-a209-35b6c05443cb None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:12.212370 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa334447-08a6-48fe-a209-35b6c05443cb None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:12.263245 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:12.270411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c9b24-ffaa-47e1-82a8-eb84db411b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:12.271734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c9b24-ffaa-47e1-82a8-eb84db411b3d admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:12.272018 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c9b24-ffaa-47e1-82a8-eb84db411b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:12.272339 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c9b24-ffaa-47e1-82a8-eb84db411b3d admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:12.272964 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c9b24-ffaa-47e1-82a8-eb84db411b3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:12.274934 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c9b24-ffaa-47e1-82a8-eb84db411b3d admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:12.298021 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1137/2298] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:59:12 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:59:12.302214 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d585d2-1e7e-4482-b044-da81a8ab55b4 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:12.312605 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d585d2-1e7e-4482-b044-da81a8ab55b4 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:12.317208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4c141-620d-4aee-b41c-49ce872bfd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:12.318490 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4c141-620d-4aee-b41c-49ce872bfd64 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:12.318775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4c141-620d-4aee-b41c-49ce872bfd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:12.319053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4c141-620d-4aee-b41c-49ce872bfd64 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:12.319693 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4c141-620d-4aee-b41c-49ce872bfd64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:12.321388 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4c141-620d-4aee-b41c-49ce872bfd64 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:12.331905 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1162/2299] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:59:12.336529 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c9b24-ffaa-47e1-82a8-eb84db411b3d admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:12.346842 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c9b24-ffaa-47e1-82a8-eb84db411b3d admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:12.351949 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42cafb8-3906-4aee-aaef-f113bff94320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:12.352659 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cafb8-3906-4aee-aaef-f113bff94320 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:12.352951 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cafb8-3906-4aee-aaef-f113bff94320 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:12.353254 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cafb8-3906-4aee-aaef-f113bff94320 admin admin] PATH_INFO: `/v3/projects/2850cc2cd73d4fcbb6b6bfcdb8952396/users/6ec2f498c3ab439cb50f9b0b8c8438ce/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:12.368110 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42cafb8-3906-4aee-aaef-f113bff94320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2850cc2cd73d4fcbb6b6bfcdb8952396, user_id=6ec2f498c3ab439cb50f9b0b8c8438ce, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:12.370790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42cafb8-3906-4aee-aaef-f113bff94320 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:12.384221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1138/2300] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:59:12 2022] PUT /identity/v3/projects/2850cc2cd73d4fcbb6b6bfcdb8952396/users/6ec2f498c3ab439cb50f9b0b8c8438ce/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:12.388272 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4c141-620d-4aee-b41c-49ce872bfd64 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:12.398702 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4c141-620d-4aee-b41c-49ce872bfd64 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:12.455144 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39236f51-3df4-4326-bc59-7c4c33d35be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:12.455737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39236f51-3df4-4326-bc59-7c4c33d35be9 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:12.456012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39236f51-3df4-4326-bc59-7c4c33d35be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:12.456313 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39236f51-3df4-4326-bc59-7c4c33d35be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:12.456939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39236f51-3df4-4326-bc59-7c4c33d35be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:12.459084 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39236f51-3df4-4326-bc59-7c4c33d35be9 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:12.469326 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1163/2301] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:59:12.473370 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42cafb8-3906-4aee-aaef-f113bff94320 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:12.484890 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42cafb8-3906-4aee-aaef-f113bff94320 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:12.490472 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f2be4-6bb5-4ede-8d57-6a156cf5a50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:12.491118 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f2be4-6bb5-4ede-8d57-6a156cf5a50e admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:12.491397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f2be4-6bb5-4ede-8d57-6a156cf5a50e admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:12.491670 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f2be4-6bb5-4ede-8d57-6a156cf5a50e admin admin] PATH_INFO: `/v3/projects/2850cc2cd73d4fcbb6b6bfcdb8952396/users/6ec2f498c3ab439cb50f9b0b8c8438ce/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:12.493606 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f2be4-6bb5-4ede-8d57-6a156cf5a50e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2850cc2cd73d4fcbb6b6bfcdb8952396, user_id=6ec2f498c3ab439cb50f9b0b8c8438ce, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:12.496333 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f2be4-6bb5-4ede-8d57-6a156cf5a50e admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:12.507067 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1139/2302] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:59:12 2022] PUT /identity/v3/projects/2850cc2cd73d4fcbb6b6bfcdb8952396/users/6ec2f498c3ab439cb50f9b0b8c8438ce/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:12.519369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c66fa-5396-461d-bdac-3d257492bdd4 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:12.519656 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c66fa-5396-461d-bdac-3d257492bdd4 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:12.519960 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c66fa-5396-461d-bdac-3d257492bdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:12.550391 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-8b9c66fa-5396-461d-bdac-3d257492bdd4 None None] MFA Rules not processed for user `6ec2f498c3ab439cb50f9b0b8c8438ce`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:12.551615 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-8b9c66fa-5396-461d-bdac-3d257492bdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:12.619966 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1164/2303] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:12 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:12.632045 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f2be4-6bb5-4ede-8d57-6a156cf5a50e admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:12.633916 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-161f2be4-6bb5-4ede-8d57-6a156cf5a50e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:12.692003 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f2be4-6bb5-4ede-8d57-6a156cf5a50e admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:12.695755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94c784-d282-4ad3-b81e-709fe74a40c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:12.696403 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94c784-d282-4ad3-b81e-709fe74a40c0 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:12.696680 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94c784-d282-4ad3-b81e-709fe74a40c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:12.696952 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94c784-d282-4ad3-b81e-709fe74a40c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:12.705871 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df94c784-d282-4ad3-b81e-709fe74a40c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:12.707317 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df94c784-d282-4ad3-b81e-709fe74a40c0 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:12.717673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1140/2304] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:12 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:59:13.143553 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c66fa-5396-461d-bdac-3d257492bdd4 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:13.169514 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:13.216873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c66fa-5396-461d-bdac-3d257492bdd4 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:13.221376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973963d-977e-4554-bfbd-3fd603df0046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:13.221998 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973963d-977e-4554-bfbd-3fd603df0046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:13.222273 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973963d-977e-4554-bfbd-3fd603df0046 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:13.222540 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973963d-977e-4554-bfbd-3fd603df0046 admin admin] PATH_INFO: `/v3/users/1ca6e5391e2348f9b8985b383742972e` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:13.223635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973963d-977e-4554-bfbd-3fd603df0046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ca6e5391e2348f9b8985b383742972e)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:13.225673 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973963d-977e-4554-bfbd-3fd603df0046 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:13.272503 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f973963d-977e-4554-bfbd-3fd603df0046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ca6e5391e2348f9b8985b383742972e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:13.273145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f973963d-977e-4554-bfbd-3fd603df0046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ca6e5391e2348f9b8985b383742972e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:13.284990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f973963d-977e-4554-bfbd-3fd603df0046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ca6e5391e2348f9b8985b383742972e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:13.292014 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f973963d-977e-4554-bfbd-3fd603df0046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ca6e5391e2348f9b8985b383742972e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:13.298426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-f973963d-977e-4554-bfbd-3fd603df0046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ca6e5391e2348f9b8985b383742972e'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:13.304406 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1165/2305] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:59:13 2022] DELETE /identity/v3/users/1ca6e5391e2348f9b8985b383742972e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:13.308942 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56e4f7-6b2b-4b65-a8f4-c0177aa85e26 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:13.309236 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56e4f7-6b2b-4b65-a8f4-c0177aa85e26 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:13.309513 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56e4f7-6b2b-4b65-a8f4-c0177aa85e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:13.328963 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-3d56e4f7-6b2b-4b65-a8f4-c0177aa85e26 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:13.330172 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-3d56e4f7-6b2b-4b65-a8f4-c0177aa85e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:13.389607 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1141/2306] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:13.400558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973963d-977e-4554-bfbd-3fd603df0046 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:13.401967 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f973963d-977e-4554-bfbd-3fd603df0046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:13.458018 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973963d-977e-4554-bfbd-3fd603df0046 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:13.462471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd97b96-0073-4271-a0d1-362c0c550a55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:13.463089 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd97b96-0073-4271-a0d1-362c0c550a55 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:13.463370 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd97b96-0073-4271-a0d1-362c0c550a55 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:13.463642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd97b96-0073-4271-a0d1-362c0c550a55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:13.473790 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd97b96-0073-4271-a0d1-362c0c550a55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:13.476051 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd97b96-0073-4271-a0d1-362c0c550a55 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:13.486174 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1166/2307] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:59:13.913373 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56e4f7-6b2b-4b65-a8f4-c0177aa85e26 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:13.914844 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-3d56e4f7-6b2b-4b65-a8f4-c0177aa85e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:13.927080 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56e4f7-6b2b-4b65-a8f4-c0177aa85e26 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:13.932034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599c192-fbc9-4519-a0b4-9fd83c2b32c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:13.932715 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599c192-fbc9-4519-a0b4-9fd83c2b32c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:13.933001 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599c192-fbc9-4519-a0b4-9fd83c2b32c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:13.933282 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599c192-fbc9-4519-a0b4-9fd83c2b32c7 admin admin] PATH_INFO: `/v3/projects/3570a646af4c45398376695d79e1b4c3` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:13.934414 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599c192-fbc9-4519-a0b4-9fd83c2b32c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3570a646af4c45398376695d79e1b4c3)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:13.936379 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599c192-fbc9-4519-a0b4-9fd83c2b32c7 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:13.998543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:14.000062 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-0599c192-fbc9-4519-a0b4-9fd83c2b32c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3570a646af4c45398376695d79e1b4c3', 'request_id': 'req-0599c192-fbc9-4519-a0b4-9fd83c2b32c7'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:14.009098 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-0599c192-fbc9-4519-a0b4-9fd83c2b32c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3570a646af4c45398376695d79e1b4c3', 'request_id': 'req-0599c192-fbc9-4519-a0b4-9fd83c2b32c7'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:14.015939 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-0599c192-fbc9-4519-a0b4-9fd83c2b32c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3570a646af4c45398376695d79e1b4c3', 'request_id': 'req-0599c192-fbc9-4519-a0b4-9fd83c2b32c7'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:14.027511 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-0599c192-fbc9-4519-a0b4-9fd83c2b32c7 admin admin] The token cache is being invalidate because project 3570a646af4c45398376695d79e1b4c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:59:14.027822 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-0599c192-fbc9-4519-a0b4-9fd83c2b32c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:14.028773 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-0599c192-fbc9-4519-a0b4-9fd83c2b32c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:14.030713 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1142/2308] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:59:13 2022] DELETE /identity/v3/projects/3570a646af4c45398376695d79e1b4c3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:14.034816 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd97b96-0073-4271-a0d1-362c0c550a55 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:14.045518 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd97b96-0073-4271-a0d1-362c0c550a55 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:14.088043 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:14.104565 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00d267-a127-4ea0-9449-6bfefc136abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:14.105201 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00d267-a127-4ea0-9449-6bfefc136abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:14.105500 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00d267-a127-4ea0-9449-6bfefc136abf admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:14.105755 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00d267-a127-4ea0-9449-6bfefc136abf admin admin] PATH_INFO: `/v3/users/6ec2f498c3ab439cb50f9b0b8c8438ce` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:14.106870 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00d267-a127-4ea0-9449-6bfefc136abf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ec2f498c3ab439cb50f9b0b8c8438ce)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:14.108834 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00d267-a127-4ea0-9449-6bfefc136abf admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:14.158031 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-9c00d267-a127-4ea0-9449-6bfefc136abf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ec2f498c3ab439cb50f9b0b8c8438ce'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:14.158667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-9c00d267-a127-4ea0-9449-6bfefc136abf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ec2f498c3ab439cb50f9b0b8c8438ce'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:14.171044 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-9c00d267-a127-4ea0-9449-6bfefc136abf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ec2f498c3ab439cb50f9b0b8c8438ce'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:14.178505 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-9c00d267-a127-4ea0-9449-6bfefc136abf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ec2f498c3ab439cb50f9b0b8c8438ce'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:14.185754 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-9c00d267-a127-4ea0-9449-6bfefc136abf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ec2f498c3ab439cb50f9b0b8c8438ce'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:14.191625 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1167/2309] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:59:14 2022] DELETE /identity/v3/users/6ec2f498c3ab439cb50f9b0b8c8438ce => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:14.539785 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599c192-fbc9-4519-a0b4-9fd83c2b32c7 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:14.541284 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-0599c192-fbc9-4519-a0b4-9fd83c2b32c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:14.555538 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599c192-fbc9-4519-a0b4-9fd83c2b32c7 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:14.615111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d516658-2c78-47e0-91e0-d3a22a0ee360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:14.615853 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d516658-2c78-47e0-91e0-d3a22a0ee360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:14.616163 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d516658-2c78-47e0-91e0-d3a22a0ee360 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:14.616446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d516658-2c78-47e0-91e0-d3a22a0ee360 admin admin] PATH_INFO: `/v3/projects/2850cc2cd73d4fcbb6b6bfcdb8952396` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:14.618318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d516658-2c78-47e0-91e0-d3a22a0ee360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2850cc2cd73d4fcbb6b6bfcdb8952396)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:14.620283 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d516658-2c78-47e0-91e0-d3a22a0ee360 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:14.669209 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-4d516658-2c78-47e0-91e0-d3a22a0ee360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2850cc2cd73d4fcbb6b6bfcdb8952396', 'request_id': 'req-4d516658-2c78-47e0-91e0-d3a22a0ee360'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:14.678711 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-4d516658-2c78-47e0-91e0-d3a22a0ee360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2850cc2cd73d4fcbb6b6bfcdb8952396', 'request_id': 'req-4d516658-2c78-47e0-91e0-d3a22a0ee360'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:14.688409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-4d516658-2c78-47e0-91e0-d3a22a0ee360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2850cc2cd73d4fcbb6b6bfcdb8952396', 'request_id': 'req-4d516658-2c78-47e0-91e0-d3a22a0ee360'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:14.710447 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-4d516658-2c78-47e0-91e0-d3a22a0ee360 admin admin] The token cache is being invalidate because project 2850cc2cd73d4fcbb6b6bfcdb8952396 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:59:14.711038 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-4d516658-2c78-47e0-91e0-d3a22a0ee360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:14.711986 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-4d516658-2c78-47e0-91e0-d3a22a0ee360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:14.714317 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1143/2310] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:59:14 2022] DELETE /identity/v3/projects/2850cc2cd73d4fcbb6b6bfcdb8952396 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:14.719411 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8efa78-eff8-44df-a3da-c324bd5b0371 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:14.719700 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8efa78-eff8-44df-a3da-c324bd5b0371 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:14.719972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8efa78-eff8-44df-a3da-c324bd5b0371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:14.745568 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-1d8efa78-eff8-44df-a3da-c324bd5b0371 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:14.746596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-1d8efa78-eff8-44df-a3da-c324bd5b0371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:14.824254 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1168/2311] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:14.828027 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afd4d2-b389-40f2-b28e-61f488ef5b7c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:14.828351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afd4d2-b389-40f2-b28e-61f488ef5b7c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:14.828598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afd4d2-b389-40f2-b28e-61f488ef5b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:14.854363 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-e5afd4d2-b389-40f2-b28e-61f488ef5b7c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:14.855631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e5afd4d2-b389-40f2-b28e-61f488ef5b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:14.865865 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1144/2312] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:14.870137 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8efa78-eff8-44df-a3da-c324bd5b0371 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:14.880130 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8efa78-eff8-44df-a3da-c324bd5b0371 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:14.884857 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2030b-582b-4b9b-8a1e-f6e831674c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:14.885465 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2030b-582b-4b9b-8a1e-f6e831674c6b admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:14.885741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2030b-582b-4b9b-8a1e-f6e831674c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:14.886012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2030b-582b-4b9b-8a1e-f6e831674c6b admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:14.886652 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2030b-582b-4b9b-8a1e-f6e831674c6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:59:14.886953 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2030b-582b-4b9b-8a1e-f6e831674c6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:14.889007 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2030b-582b-4b9b-8a1e-f6e831674c6b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:14.903323 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1169/2313] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:59:14.907810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afd4d2-b389-40f2-b28e-61f488ef5b7c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:14.918457 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afd4d2-b389-40f2-b28e-61f488ef5b7c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:14.923132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd460c-a309-45f3-8ed2-89f60d8d61f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:14.923689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd460c-a309-45f3-8ed2-89f60d8d61f9 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:14.923952 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd460c-a309-45f3-8ed2-89f60d8d61f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:14.924378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd460c-a309-45f3-8ed2-89f60d8d61f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:14.925007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd460c-a309-45f3-8ed2-89f60d8d61f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:14.927188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd460c-a309-45f3-8ed2-89f60d8d61f9 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:14.955034 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1145/2314] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:59:14 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:59:14.959815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2030b-582b-4b9b-8a1e-f6e831674c6b admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:14.970630 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2030b-582b-4b9b-8a1e-f6e831674c6b admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:15.019667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:15.032517 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc30b3e-7889-413f-9f4e-f8911ee8fcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:15.033122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc30b3e-7889-413f-9f4e-f8911ee8fcc3 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:15.033399 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc30b3e-7889-413f-9f4e-f8911ee8fcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:15.033667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc30b3e-7889-413f-9f4e-f8911ee8fcc3 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:15.034288 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc30b3e-7889-413f-9f4e-f8911ee8fcc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:15.036325 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc30b3e-7889-413f-9f4e-f8911ee8fcc3 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:15.058330 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1170/2315] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:59:14 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:59:15.062467 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd460c-a309-45f3-8ed2-89f60d8d61f9 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:15.073031 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd460c-a309-45f3-8ed2-89f60d8d61f9 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:15.077651 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1c27c-d7f0-4d5f-9ed3-ffd3f543cf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:15.078252 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1c27c-d7f0-4d5f-9ed3-ffd3f543cf04 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:15.078530 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1c27c-d7f0-4d5f-9ed3-ffd3f543cf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:15.078807 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1c27c-d7f0-4d5f-9ed3-ffd3f543cf04 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:15.079442 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1c27c-d7f0-4d5f-9ed3-ffd3f543cf04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:15.081076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1c27c-d7f0-4d5f-9ed3-ffd3f543cf04 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:15.092288 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1146/2316] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:59:15.096365 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc30b3e-7889-413f-9f4e-f8911ee8fcc3 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:15.107147 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc30b3e-7889-413f-9f4e-f8911ee8fcc3 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:15.111790 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61efebb2-7b1b-469b-a60b-e097fb06bf68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:15.112418 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efebb2-7b1b-469b-a60b-e097fb06bf68 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:15.112703 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efebb2-7b1b-469b-a60b-e097fb06bf68 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:15.112977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efebb2-7b1b-469b-a60b-e097fb06bf68 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:15.113611 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61efebb2-7b1b-469b-a60b-e097fb06bf68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:15.115317 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61efebb2-7b1b-469b-a60b-e097fb06bf68 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:15.126232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1171/2317] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:59:15.130355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1c27c-d7f0-4d5f-9ed3-ffd3f543cf04 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:15.140291 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1c27c-d7f0-4d5f-9ed3-ffd3f543cf04 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:15.144880 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60294b0-07cc-4cc9-930d-4b4bc1792cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:15.145525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60294b0-07cc-4cc9-930d-4b4bc1792cc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:15.146297 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60294b0-07cc-4cc9-930d-4b4bc1792cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:15.146575 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60294b0-07cc-4cc9-930d-4b4bc1792cc6 admin admin] PATH_INFO: `/v3/projects/6d1a324efad74bf9a86984b1db9f56a4/users/213e82a8bc0a4fcb96ad69c3a01b3597/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:15.161147 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60294b0-07cc-4cc9-930d-4b4bc1792cc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d1a324efad74bf9a86984b1db9f56a4, user_id=213e82a8bc0a4fcb96ad69c3a01b3597, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:15.164517 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60294b0-07cc-4cc9-930d-4b4bc1792cc6 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:15.177530 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1147/2318] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:59:15 2022] PUT /identity/v3/projects/6d1a324efad74bf9a86984b1db9f56a4/users/213e82a8bc0a4fcb96ad69c3a01b3597/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:15.187327 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c302c93-c2f8-40d2-b502-bb8833899e03 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:15.187812 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c302c93-c2f8-40d2-b502-bb8833899e03 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:15.187935 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c302c93-c2f8-40d2-b502-bb8833899e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:15.215940 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-5c302c93-c2f8-40d2-b502-bb8833899e03 None None] MFA Rules not processed for user `213e82a8bc0a4fcb96ad69c3a01b3597`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:15.217448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-5c302c93-c2f8-40d2-b502-bb8833899e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:15.284375 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1172/2319] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:15.289180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60294b0-07cc-4cc9-930d-4b4bc1792cc6 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:15.293953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:15.355591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60294b0-07cc-4cc9-930d-4b4bc1792cc6 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:15.360478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a996450-4a69-427a-bcf7-9db601f678f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:15.361084 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a996450-4a69-427a-bcf7-9db601f678f9 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:15.361365 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a996450-4a69-427a-bcf7-9db601f678f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:15.361636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a996450-4a69-427a-bcf7-9db601f678f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:15.362276 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a996450-4a69-427a-bcf7-9db601f678f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:15.364210 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a996450-4a69-427a-bcf7-9db601f678f9 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:15.391014 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1148/2320] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:59:15 2022] POST /identity/v3/projects => generated 391 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:59:15.395236 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c302c93-c2f8-40d2-b502-bb8833899e03 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:15.406035 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c302c93-c2f8-40d2-b502-bb8833899e03 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:15.463655 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09806303-4feb-454b-ad43-7008fbcbf757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:15.464296 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09806303-4feb-454b-ad43-7008fbcbf757 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:15.464587 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09806303-4feb-454b-ad43-7008fbcbf757 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:15.464864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09806303-4feb-454b-ad43-7008fbcbf757 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:15.465519 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09806303-4feb-454b-ad43-7008fbcbf757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:15.467758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09806303-4feb-454b-ad43-7008fbcbf757 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:15.491091 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1173/2321] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:59:15 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:59:15.495141 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a996450-4a69-427a-bcf7-9db601f678f9 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:15.506177 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a996450-4a69-427a-bcf7-9db601f678f9 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:15.511464 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab8e2f-e671-444e-afb8-3c5b552bdd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:15.512165 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab8e2f-e671-444e-afb8-3c5b552bdd56 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:15.512475 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab8e2f-e671-444e-afb8-3c5b552bdd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:15.512746 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab8e2f-e671-444e-afb8-3c5b552bdd56 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:15.513379 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab8e2f-e671-444e-afb8-3c5b552bdd56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:15.514999 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab8e2f-e671-444e-afb8-3c5b552bdd56 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:15.526486 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1149/2322] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:59:15.530739 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09806303-4feb-454b-ad43-7008fbcbf757 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:15.541279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09806303-4feb-454b-ad43-7008fbcbf757 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:15.545867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff75962-1e8d-4650-a2dc-96ee0b749772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:15.546525 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff75962-1e8d-4650-a2dc-96ee0b749772 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:15.546798 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff75962-1e8d-4650-a2dc-96ee0b749772 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:15.547065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff75962-1e8d-4650-a2dc-96ee0b749772 admin admin] PATH_INFO: `/v3/projects/f835b35659674bd38d5b54bbf3bf8bca/users/2d249d13e9804a289fcd4739013280ea/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:15.562707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff75962-1e8d-4650-a2dc-96ee0b749772 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f835b35659674bd38d5b54bbf3bf8bca, user_id=2d249d13e9804a289fcd4739013280ea, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:15.565427 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff75962-1e8d-4650-a2dc-96ee0b749772 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:15.580104 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1174/2323] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:59:15 2022] PUT /identity/v3/projects/f835b35659674bd38d5b54bbf3bf8bca/users/2d249d13e9804a289fcd4739013280ea/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:59:15.584313 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab8e2f-e671-444e-afb8-3c5b552bdd56 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:15.594896 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab8e2f-e671-444e-afb8-3c5b552bdd56 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:15.662375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c691dbe-3add-40d9-adc0-ed5d522d5224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:15.662699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c691dbe-3add-40d9-adc0-ed5d522d5224 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:15.662986 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c691dbe-3add-40d9-adc0-ed5d522d5224 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:15.663259 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c691dbe-3add-40d9-adc0-ed5d522d5224 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:15.663892 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c691dbe-3add-40d9-adc0-ed5d522d5224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:15.665546 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c691dbe-3add-40d9-adc0-ed5d522d5224 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:15.675901 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1150/2324] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:59:15.680181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff75962-1e8d-4650-a2dc-96ee0b749772 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:15.690592 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff75962-1e8d-4650-a2dc-96ee0b749772 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:15.695376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a7807-7ecf-4fe6-b871-937c1de70ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:15.696282 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a7807-7ecf-4fe6-b871-937c1de70ca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:15.696622 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a7807-7ecf-4fe6-b871-937c1de70ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:15.696901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a7807-7ecf-4fe6-b871-937c1de70ca9 admin admin] PATH_INFO: `/v3/projects/f835b35659674bd38d5b54bbf3bf8bca/users/2d249d13e9804a289fcd4739013280ea/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:15.698931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a7807-7ecf-4fe6-b871-937c1de70ca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f835b35659674bd38d5b54bbf3bf8bca, user_id=2d249d13e9804a289fcd4739013280ea, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:15.701716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a7807-7ecf-4fe6-b871-937c1de70ca9 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:15.713254 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1175/2325] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:59:15 2022] PUT /identity/v3/projects/f835b35659674bd38d5b54bbf3bf8bca/users/2d249d13e9804a289fcd4739013280ea/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:15.724661 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2ed52-fd00-4bc9-8e3f-03091913280e None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:15.724941 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2ed52-fd00-4bc9-8e3f-03091913280e None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:15.725331 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2ed52-fd00-4bc9-8e3f-03091913280e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:15.753077 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-43a2ed52-fd00-4bc9-8e3f-03091913280e None None] MFA Rules not processed for user `2d249d13e9804a289fcd4739013280ea`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:15.754296 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-43a2ed52-fd00-4bc9-8e3f-03091913280e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:15.816330 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:15.822619 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1151/2326] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:15 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:15.833962 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a7807-7ecf-4fe6-b871-937c1de70ca9 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:15.835641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-bc2a7807-7ecf-4fe6-b871-937c1de70ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:15.850406 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:15.896010 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a7807-7ecf-4fe6-b871-937c1de70ca9 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:15.899654 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe92705-2115-4fd1-a193-b8a3b0e17459 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:15.900299 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe92705-2115-4fd1-a193-b8a3b0e17459 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:15.900578 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe92705-2115-4fd1-a193-b8a3b0e17459 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:15.900851 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe92705-2115-4fd1-a193-b8a3b0e17459 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:15.909762 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe92705-2115-4fd1-a193-b8a3b0e17459 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:15.911184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe92705-2115-4fd1-a193-b8a3b0e17459 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:15.920508 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1176/2327] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:15 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:59:15.968013 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2ed52-fd00-4bc9-8e3f-03091913280e None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:15.969365 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-43a2ed52-fd00-4bc9-8e3f-03091913280e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:15.982011 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2ed52-fd00-4bc9-8e3f-03091913280e None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:15.985679 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383ba63-d341-4b97-96e9-d580f6581fd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:15.986279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383ba63-d341-4b97-96e9-d580f6581fd6 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:15.986557 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383ba63-d341-4b97-96e9-d580f6581fd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:15.986829 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383ba63-d341-4b97-96e9-d580f6581fd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:16.044146 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383ba63-d341-4b97-96e9-d580f6581fd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:16.045590 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383ba63-d341-4b97-96e9-d580f6581fd6 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:16.054474 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1152/2328] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:59:19.348989 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe92705-2115-4fd1-a193-b8a3b0e17459 service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:19.426416 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe92705-2115-4fd1-a193-b8a3b0e17459 service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:19.432916 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5de5aa-d45d-4749-a110-d6b2342fc606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:19.433565 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5de5aa-d45d-4749-a110-d6b2342fc606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:19.433839 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5de5aa-d45d-4749-a110-d6b2342fc606 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:19.434109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5de5aa-d45d-4749-a110-d6b2342fc606 admin admin] PATH_INFO: `/v3/users/213e82a8bc0a4fcb96ad69c3a01b3597` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:19.435365 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5de5aa-d45d-4749-a110-d6b2342fc606 admin admin] RBAC: Authorizing `identity:delete_user(user_id=213e82a8bc0a4fcb96ad69c3a01b3597)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:19.437333 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5de5aa-d45d-4749-a110-d6b2342fc606 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:19.498865 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-9e5de5aa-d45d-4749-a110-d6b2342fc606 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '213e82a8bc0a4fcb96ad69c3a01b3597'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:19.500214 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-9e5de5aa-d45d-4749-a110-d6b2342fc606 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '213e82a8bc0a4fcb96ad69c3a01b3597'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:19.519024 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-9e5de5aa-d45d-4749-a110-d6b2342fc606 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '213e82a8bc0a4fcb96ad69c3a01b3597'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:19.526171 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-9e5de5aa-d45d-4749-a110-d6b2342fc606 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '213e82a8bc0a4fcb96ad69c3a01b3597'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:19.534718 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-9e5de5aa-d45d-4749-a110-d6b2342fc606 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '213e82a8bc0a4fcb96ad69c3a01b3597'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:19.541142 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1177/2329] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:59:19 2022] DELETE /identity/v3/users/213e82a8bc0a4fcb96ad69c3a01b3597 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:19.545611 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a38e6f-94d4-4323-9335-a5817e7c98a2 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:19.546073 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a38e6f-94d4-4323-9335-a5817e7c98a2 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:19.546392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a38e6f-94d4-4323-9335-a5817e7c98a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:19.566828 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-45a38e6f-94d4-4323-9335-a5817e7c98a2 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:19.568071 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-45a38e6f-94d4-4323-9335-a5817e7c98a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:19.630045 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1153/2330] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:19.640285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5de5aa-d45d-4749-a110-d6b2342fc606 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:19.641809 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-9e5de5aa-d45d-4749-a110-d6b2342fc606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:19.677782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:19.704275 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5de5aa-d45d-4749-a110-d6b2342fc606 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:19.707911 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e98cc4-cea0-44ae-bf95-a15cb0ba1a09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:19.708571 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e98cc4-cea0-44ae-bf95-a15cb0ba1a09 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:19.708851 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e98cc4-cea0-44ae-bf95-a15cb0ba1a09 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:19.709124 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e98cc4-cea0-44ae-bf95-a15cb0ba1a09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:19.719668 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e98cc4-cea0-44ae-bf95-a15cb0ba1a09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:19.721156 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e98cc4-cea0-44ae-bf95-a15cb0ba1a09 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:19.731880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1178/2331] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:59:20.017984 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a38e6f-94d4-4323-9335-a5817e7c98a2 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:20.019478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-45a38e6f-94d4-4323-9335-a5817e7c98a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:20.036735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a38e6f-94d4-4323-9335-a5817e7c98a2 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:20.043398 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97850ab5-80b9-4015-9772-25c66e571b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:20.044182 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97850ab5-80b9-4015-9772-25c66e571b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:20.044492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97850ab5-80b9-4015-9772-25c66e571b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:20.044769 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97850ab5-80b9-4015-9772-25c66e571b0b admin admin] PATH_INFO: `/v3/projects/6d1a324efad74bf9a86984b1db9f56a4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:20.046118 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97850ab5-80b9-4015-9772-25c66e571b0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d1a324efad74bf9a86984b1db9f56a4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:20.048459 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97850ab5-80b9-4015-9772-25c66e571b0b admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:20.100866 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-97850ab5-80b9-4015-9772-25c66e571b0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d1a324efad74bf9a86984b1db9f56a4', 'request_id': 'req-97850ab5-80b9-4015-9772-25c66e571b0b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:20.108811 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-97850ab5-80b9-4015-9772-25c66e571b0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d1a324efad74bf9a86984b1db9f56a4', 'request_id': 'req-97850ab5-80b9-4015-9772-25c66e571b0b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:20.115159 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-97850ab5-80b9-4015-9772-25c66e571b0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d1a324efad74bf9a86984b1db9f56a4', 'request_id': 'req-97850ab5-80b9-4015-9772-25c66e571b0b'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:20.128093 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-97850ab5-80b9-4015-9772-25c66e571b0b admin admin] The token cache is being invalidate because project 6d1a324efad74bf9a86984b1db9f56a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:59:20.128395 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-97850ab5-80b9-4015-9772-25c66e571b0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:20.129043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-97850ab5-80b9-4015-9772-25c66e571b0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:20.130525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1154/2332] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:59:20 2022] DELETE /identity/v3/projects/6d1a324efad74bf9a86984b1db9f56a4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:20.135260 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e98cc4-cea0-44ae-bf95-a15cb0ba1a09 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:20.144854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e98cc4-cea0-44ae-bf95-a15cb0ba1a09 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:20.204759 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d16b2-28c9-48d0-90d4-220d29c3b495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:20.205398 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d16b2-28c9-48d0-90d4-220d29c3b495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:20.205679 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d16b2-28c9-48d0-90d4-220d29c3b495 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:20.205952 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d16b2-28c9-48d0-90d4-220d29c3b495 admin admin] PATH_INFO: `/v3/users/2d249d13e9804a289fcd4739013280ea` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:20.207090 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d16b2-28c9-48d0-90d4-220d29c3b495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d249d13e9804a289fcd4739013280ea)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:20.209106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d16b2-28c9-48d0-90d4-220d29c3b495 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:20.256822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-018d16b2-28c9-48d0-90d4-220d29c3b495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d249d13e9804a289fcd4739013280ea'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:20.258081 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-018d16b2-28c9-48d0-90d4-220d29c3b495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d249d13e9804a289fcd4739013280ea'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:20.270035 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-018d16b2-28c9-48d0-90d4-220d29c3b495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d249d13e9804a289fcd4739013280ea'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:20.277575 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-018d16b2-28c9-48d0-90d4-220d29c3b495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d249d13e9804a289fcd4739013280ea'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:20.286147 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-018d16b2-28c9-48d0-90d4-220d29c3b495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d249d13e9804a289fcd4739013280ea'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:20.292439 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1179/2333] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:59:20 2022] DELETE /identity/v3/users/2d249d13e9804a289fcd4739013280ea => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:20.964887 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97850ab5-80b9-4015-9772-25c66e571b0b admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:20.966709 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-97850ab5-80b9-4015-9772-25c66e571b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:20.982900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97850ab5-80b9-4015-9772-25c66e571b0b admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:21.002849 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:21.053000 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d626c-973e-4540-8091-bee238e25330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:21.053756 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d626c-973e-4540-8091-bee238e25330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:21.054038 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d626c-973e-4540-8091-bee238e25330 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:21.055024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d626c-973e-4540-8091-bee238e25330 admin admin] PATH_INFO: `/v3/projects/f835b35659674bd38d5b54bbf3bf8bca` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:21.056545 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d626c-973e-4540-8091-bee238e25330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f835b35659674bd38d5b54bbf3bf8bca)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:21.058467 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d626c-973e-4540-8091-bee238e25330 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:21.120434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-502d626c-973e-4540-8091-bee238e25330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f835b35659674bd38d5b54bbf3bf8bca', 'request_id': 'req-502d626c-973e-4540-8091-bee238e25330'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:21.133447 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-502d626c-973e-4540-8091-bee238e25330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f835b35659674bd38d5b54bbf3bf8bca', 'request_id': 'req-502d626c-973e-4540-8091-bee238e25330'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:21.141224 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-502d626c-973e-4540-8091-bee238e25330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f835b35659674bd38d5b54bbf3bf8bca', 'request_id': 'req-502d626c-973e-4540-8091-bee238e25330'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:21.154920 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-502d626c-973e-4540-8091-bee238e25330 admin admin] The token cache is being invalidate because project f835b35659674bd38d5b54bbf3bf8bca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:59:21.155217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-502d626c-973e-4540-8091-bee238e25330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:21.155885 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-502d626c-973e-4540-8091-bee238e25330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:21.160722 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1155/2334] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:59:20 2022] DELETE /identity/v3/projects/f835b35659674bd38d5b54bbf3bf8bca => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:21.162431 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9448aa0-e23d-4ee1-bc6a-ae1396e73b5a None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:21.162931 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9448aa0-e23d-4ee1-bc6a-ae1396e73b5a None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:21.163064 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9448aa0-e23d-4ee1-bc6a-ae1396e73b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:21.195887 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-f9448aa0-e23d-4ee1-bc6a-ae1396e73b5a None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:21.197475 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f9448aa0-e23d-4ee1-bc6a-ae1396e73b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:21.256688 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:21.280281 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1180/2335] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:21.282749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721493f-dc69-4ae7-ac47-d4e67e89b555 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:21.283039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721493f-dc69-4ae7-ac47-d4e67e89b555 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:21.283798 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721493f-dc69-4ae7-ac47-d4e67e89b555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:21.313620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-8721493f-dc69-4ae7-ac47-d4e67e89b555 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:21.314858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-8721493f-dc69-4ae7-ac47-d4e67e89b555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:21.323968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1156/2336] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 00:59:21.328270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9448aa0-e23d-4ee1-bc6a-ae1396e73b5a None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:21.339450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9448aa0-e23d-4ee1-bc6a-ae1396e73b5a None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:21.345790 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c5c05-a4f3-46d2-a10d-72513de1fe12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:21.346431 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c5c05-a4f3-46d2-a10d-72513de1fe12 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:21.346706 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c5c05-a4f3-46d2-a10d-72513de1fe12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:21.346983 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c5c05-a4f3-46d2-a10d-72513de1fe12 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:21.347660 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c5c05-a4f3-46d2-a10d-72513de1fe12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:59:21.347965 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c5c05-a4f3-46d2-a10d-72513de1fe12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:21.349880 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c5c05-a4f3-46d2-a10d-72513de1fe12 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:21.364121 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1181/2337] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:59:21.367473 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721493f-dc69-4ae7-ac47-d4e67e89b555 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:21.377299 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721493f-dc69-4ae7-ac47-d4e67e89b555 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:21.382002 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8c37b-9e2f-4720-b131-fc4b5283f392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:21.383266 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8c37b-9e2f-4720-b131-fc4b5283f392 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:21.383544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8c37b-9e2f-4720-b131-fc4b5283f392 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:21.383815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8c37b-9e2f-4720-b131-fc4b5283f392 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:21.384587 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8c37b-9e2f-4720-b131-fc4b5283f392 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:21.386510 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8c37b-9e2f-4720-b131-fc4b5283f392 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:21.419223 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1157/2338] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:59:21 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:59:21.420428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c5c05-a4f3-46d2-a10d-72513de1fe12 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:21.431075 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c5c05-a4f3-46d2-a10d-72513de1fe12 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:21.493827 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657a8b9-3df8-4b36-b7a7-45e853cb096a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:21.494376 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657a8b9-3df8-4b36-b7a7-45e853cb096a admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:21.494722 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657a8b9-3df8-4b36-b7a7-45e853cb096a admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:21.495008 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657a8b9-3df8-4b36-b7a7-45e853cb096a admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:21.495636 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657a8b9-3df8-4b36-b7a7-45e853cb096a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:21.497659 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657a8b9-3df8-4b36-b7a7-45e853cb096a admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:21.520580 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1182/2339] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:59:21 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:59:21.525412 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8c37b-9e2f-4720-b131-fc4b5283f392 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:21.536306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8c37b-9e2f-4720-b131-fc4b5283f392 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:21.541075 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a7539-7336-400d-afeb-c88b587a4596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:21.541695 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a7539-7336-400d-afeb-c88b587a4596 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:21.541979 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a7539-7336-400d-afeb-c88b587a4596 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:21.542284 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a7539-7336-400d-afeb-c88b587a4596 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:21.542937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a7539-7336-400d-afeb-c88b587a4596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:21.544611 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a7539-7336-400d-afeb-c88b587a4596 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:21.558474 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1158/2340] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:59:21.559766 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657a8b9-3df8-4b36-b7a7-45e853cb096a admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:21.570778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657a8b9-3df8-4b36-b7a7-45e853cb096a admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:21.575515 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a89ab-76b0-4e6d-897b-dc5b08684a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:21.576175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a89ab-76b0-4e6d-897b-dc5b08684a67 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:21.576454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a89ab-76b0-4e6d-897b-dc5b08684a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:21.576727 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a89ab-76b0-4e6d-897b-dc5b08684a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:21.577366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a89ab-76b0-4e6d-897b-dc5b08684a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:21.578979 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a89ab-76b0-4e6d-897b-dc5b08684a67 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:21.591388 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1183/2341] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:59:21.596030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a7539-7336-400d-afeb-c88b587a4596 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:21.606669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a7539-7336-400d-afeb-c88b587a4596 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:21.612019 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f2890-f30b-4fdf-a1b0-b73cf68aff9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:21.612721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f2890-f30b-4fdf-a1b0-b73cf68aff9e admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:21.612999 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f2890-f30b-4fdf-a1b0-b73cf68aff9e admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:21.613271 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f2890-f30b-4fdf-a1b0-b73cf68aff9e admin admin] PATH_INFO: `/v3/projects/ee26451f71e54e0a8a24f30ee6f4eaba/users/8b7694ef90dc42488566f350de4be367/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:21.629896 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f2890-f30b-4fdf-a1b0-b73cf68aff9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee26451f71e54e0a8a24f30ee6f4eaba, user_id=8b7694ef90dc42488566f350de4be367, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:21.633006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f2890-f30b-4fdf-a1b0-b73cf68aff9e admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:21.646703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1159/2342] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:59:21 2022] PUT /identity/v3/projects/ee26451f71e54e0a8a24f30ee6f4eaba/users/8b7694ef90dc42488566f350de4be367/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:59:21.657557 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0beefd7-e8a7-40d8-948c-fcf14677d153 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:21.657848 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0beefd7-e8a7-40d8-948c-fcf14677d153 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:21.658128 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0beefd7-e8a7-40d8-948c-fcf14677d153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:21.687976 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-b0beefd7-e8a7-40d8-948c-fcf14677d153 None None] MFA Rules not processed for user `8b7694ef90dc42488566f350de4be367`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:21.689239 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-b0beefd7-e8a7-40d8-948c-fcf14677d153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:21.762853 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1184/2343] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:21.775301 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f2890-f30b-4fdf-a1b0-b73cf68aff9e admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:21.823766 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:21.840851 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f2890-f30b-4fdf-a1b0-b73cf68aff9e admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:21.845704 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d6c83-6a59-42f2-a50b-7d8d2cd51f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:21.846303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d6c83-6a59-42f2-a50b-7d8d2cd51f63 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:21.846577 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d6c83-6a59-42f2-a50b-7d8d2cd51f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:21.846844 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d6c83-6a59-42f2-a50b-7d8d2cd51f63 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:21.847473 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d6c83-6a59-42f2-a50b-7d8d2cd51f63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:21.849396 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d6c83-6a59-42f2-a50b-7d8d2cd51f63 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:21.876851 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1160/2344] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:59:21 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 00:59:21.880884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0beefd7-e8a7-40d8-948c-fcf14677d153 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:21.891611 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0beefd7-e8a7-40d8-948c-fcf14677d153 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:21.938976 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:21.949537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56640d41-dec9-4e31-9b0e-2dfcbbc4fa25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:21.950151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56640d41-dec9-4e31-9b0e-2dfcbbc4fa25 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:21.950432 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56640d41-dec9-4e31-9b0e-2dfcbbc4fa25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:21.950708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56640d41-dec9-4e31-9b0e-2dfcbbc4fa25 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:21.951401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56640d41-dec9-4e31-9b0e-2dfcbbc4fa25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:21.953360 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56640d41-dec9-4e31-9b0e-2dfcbbc4fa25 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:21.975032 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1185/2345] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:59:21 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:59:21.979735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d6c83-6a59-42f2-a50b-7d8d2cd51f63 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:21.989911 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d6c83-6a59-42f2-a50b-7d8d2cd51f63 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:21.994787 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0dad6b-837d-4d1f-9243-621ca6671bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:21.995379 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dad6b-837d-4d1f-9243-621ca6671bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:21.995658 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dad6b-837d-4d1f-9243-621ca6671bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:21.995929 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dad6b-837d-4d1f-9243-621ca6671bf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:21.996586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0dad6b-837d-4d1f-9243-621ca6671bf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:21.998190 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0dad6b-837d-4d1f-9243-621ca6671bf5 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:22.009731 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1161/2346] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:59:22.013895 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56640d41-dec9-4e31-9b0e-2dfcbbc4fa25 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:22.024915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56640d41-dec9-4e31-9b0e-2dfcbbc4fa25 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:22.029440 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbac515-b124-4300-8429-0a7eeaeee788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:22.030083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbac515-b124-4300-8429-0a7eeaeee788 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:22.030365 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbac515-b124-4300-8429-0a7eeaeee788 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:22.030634 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbac515-b124-4300-8429-0a7eeaeee788 admin admin] PATH_INFO: `/v3/projects/13c6b43710a240edbf0c0bb4ab9c5c2f/users/72b904c98d3244d58afa185265e4e749/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:22.045473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbac515-b124-4300-8429-0a7eeaeee788 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13c6b43710a240edbf0c0bb4ab9c5c2f, user_id=72b904c98d3244d58afa185265e4e749, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:22.048135 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbac515-b124-4300-8429-0a7eeaeee788 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:22.061429 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1186/2347] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:59:22 2022] PUT /identity/v3/projects/13c6b43710a240edbf0c0bb4ab9c5c2f/users/72b904c98d3244d58afa185265e4e749/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:22.065335 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0dad6b-837d-4d1f-9243-621ca6671bf5 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:22.076289 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0dad6b-837d-4d1f-9243-621ca6671bf5 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:22.135064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f37e9b-8045-4bb1-9d6d-d540b1d82c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:22.135674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f37e9b-8045-4bb1-9d6d-d540b1d82c36 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:22.135953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f37e9b-8045-4bb1-9d6d-d540b1d82c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:22.136248 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f37e9b-8045-4bb1-9d6d-d540b1d82c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:22.136881 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f37e9b-8045-4bb1-9d6d-d540b1d82c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:22.138474 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f37e9b-8045-4bb1-9d6d-d540b1d82c36 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:22.149368 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1162/2348] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:59:22.153548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbac515-b124-4300-8429-0a7eeaeee788 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:22.164178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbac515-b124-4300-8429-0a7eeaeee788 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:22.168829 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7e812-9549-4c6b-ae6b-33557cd5a13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:22.169476 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7e812-9549-4c6b-ae6b-33557cd5a13a admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:22.169753 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7e812-9549-4c6b-ae6b-33557cd5a13a admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:22.170027 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7e812-9549-4c6b-ae6b-33557cd5a13a admin admin] PATH_INFO: `/v3/projects/13c6b43710a240edbf0c0bb4ab9c5c2f/users/72b904c98d3244d58afa185265e4e749/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:22.172061 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7e812-9549-4c6b-ae6b-33557cd5a13a admin admin] RBAC: Authorizing `identity:create_grant(project_id=13c6b43710a240edbf0c0bb4ab9c5c2f, user_id=72b904c98d3244d58afa185265e4e749, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:22.174770 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7e812-9549-4c6b-ae6b-33557cd5a13a admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:22.185357 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1187/2349] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:59:22 2022] PUT /identity/v3/projects/13c6b43710a240edbf0c0bb4ab9c5c2f/users/72b904c98d3244d58afa185265e4e749/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:22.197468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9660516-4fac-4ff9-84f1-73b741554566 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:22.197757 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9660516-4fac-4ff9-84f1-73b741554566 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:22.198033 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9660516-4fac-4ff9-84f1-73b741554566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:22.227342 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-e9660516-4fac-4ff9-84f1-73b741554566 None None] MFA Rules not processed for user `72b904c98d3244d58afa185265e4e749`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:22.228584 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e9660516-4fac-4ff9-84f1-73b741554566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:22.295844 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1163/2350] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:22 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:22.307058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7e812-9549-4c6b-ae6b-33557cd5a13a admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:22.308790 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-40a7e812-9549-4c6b-ae6b-33557cd5a13a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:22.367599 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7e812-9549-4c6b-ae6b-33557cd5a13a admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:22.371259 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7056ae6-f90d-4265-943b-ef140719636f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:22.371862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7056ae6-f90d-4265-943b-ef140719636f service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:22.372180 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7056ae6-f90d-4265-943b-ef140719636f service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:22.372454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7056ae6-f90d-4265-943b-ef140719636f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:22.381354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7056ae6-f90d-4265-943b-ef140719636f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:22.383369 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7056ae6-f90d-4265-943b-ef140719636f service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:22.392612 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1188/2351] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:59:22.464864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9660516-4fac-4ff9-84f1-73b741554566 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:22.474839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9660516-4fac-4ff9-84f1-73b741554566 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:22.478601 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f6060-543f-4bd1-9ee5-3e39f33ddf81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:22.479197 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f6060-543f-4bd1-9ee5-3e39f33ddf81 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:22.479476 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f6060-543f-4bd1-9ee5-3e39f33ddf81 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:22.479748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f6060-543f-4bd1-9ee5-3e39f33ddf81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:22.529308 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:22.557914 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f6060-543f-4bd1-9ee5-3e39f33ddf81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:22.559287 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f6060-543f-4bd1-9ee5-3e39f33ddf81 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:22.568229 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1164/2352] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:59:22.842985 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7056ae6-f90d-4265-943b-ef140719636f service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:22.905131 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7056ae6-f90d-4265-943b-ef140719636f service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:22.909633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e810d13-6d44-4f8c-8fc1-1b83f8687ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:22.910261 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e810d13-6d44-4f8c-8fc1-1b83f8687ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:22.910540 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e810d13-6d44-4f8c-8fc1-1b83f8687ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:22.910815 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e810d13-6d44-4f8c-8fc1-1b83f8687ffd admin admin] PATH_INFO: `/v3/users/8b7694ef90dc42488566f350de4be367` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:22.911920 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e810d13-6d44-4f8c-8fc1-1b83f8687ffd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b7694ef90dc42488566f350de4be367)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:22.914357 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e810d13-6d44-4f8c-8fc1-1b83f8687ffd admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:22.963170 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-0e810d13-6d44-4f8c-8fc1-1b83f8687ffd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b7694ef90dc42488566f350de4be367'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:22.963813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-0e810d13-6d44-4f8c-8fc1-1b83f8687ffd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b7694ef90dc42488566f350de4be367'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:22.964941 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:22.979176 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-0e810d13-6d44-4f8c-8fc1-1b83f8687ffd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b7694ef90dc42488566f350de4be367'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:22.986249 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-0e810d13-6d44-4f8c-8fc1-1b83f8687ffd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b7694ef90dc42488566f350de4be367'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:22.994227 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-0e810d13-6d44-4f8c-8fc1-1b83f8687ffd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b7694ef90dc42488566f350de4be367'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:23.000366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1189/2353] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:59:22 2022] DELETE /identity/v3/users/8b7694ef90dc42488566f350de4be367 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:23.005016 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb54f54-957b-4d22-ae15-1d9b50ebbd3c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:23.005547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb54f54-957b-4d22-ae15-1d9b50ebbd3c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:23.005547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb54f54-957b-4d22-ae15-1d9b50ebbd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:23.024541 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-0cb54f54-957b-4d22-ae15-1d9b50ebbd3c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:23.025753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-0cb54f54-957b-4d22-ae15-1d9b50ebbd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:23.084379 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1165/2354] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:23.094813 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e810d13-6d44-4f8c-8fc1-1b83f8687ffd admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:23.096232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-0e810d13-6d44-4f8c-8fc1-1b83f8687ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:23.153144 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e810d13-6d44-4f8c-8fc1-1b83f8687ffd admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:23.156754 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe6198-fbaf-4a3e-a8ea-f29f5e38a852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:23.157383 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe6198-fbaf-4a3e-a8ea-f29f5e38a852 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:23.157685 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe6198-fbaf-4a3e-a8ea-f29f5e38a852 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:23.157950 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe6198-fbaf-4a3e-a8ea-f29f5e38a852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:23.167558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fe6198-fbaf-4a3e-a8ea-f29f5e38a852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:23.169005 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fe6198-fbaf-4a3e-a8ea-f29f5e38a852 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:23.178145 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1190/2355] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:59:23.453431 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb54f54-957b-4d22-ae15-1d9b50ebbd3c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:23.455012 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-0cb54f54-957b-4d22-ae15-1d9b50ebbd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:23.467910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb54f54-957b-4d22-ae15-1d9b50ebbd3c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:23.472674 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bdf03-4621-4146-ab68-bbf4a6bb57c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:23.473313 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bdf03-4621-4146-ab68-bbf4a6bb57c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:23.473592 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bdf03-4621-4146-ab68-bbf4a6bb57c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:23.473905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bdf03-4621-4146-ab68-bbf4a6bb57c1 admin admin] PATH_INFO: `/v3/projects/ee26451f71e54e0a8a24f30ee6f4eaba` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:23.474966 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bdf03-4621-4146-ab68-bbf4a6bb57c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee26451f71e54e0a8a24f30ee6f4eaba)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:23.477610 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bdf03-4621-4146-ab68-bbf4a6bb57c1 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:23.528180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b98bdf03-4621-4146-ab68-bbf4a6bb57c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee26451f71e54e0a8a24f30ee6f4eaba', 'request_id': 'req-b98bdf03-4621-4146-ab68-bbf4a6bb57c1'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:23.535820 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b98bdf03-4621-4146-ab68-bbf4a6bb57c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee26451f71e54e0a8a24f30ee6f4eaba', 'request_id': 'req-b98bdf03-4621-4146-ab68-bbf4a6bb57c1'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:23.543475 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b98bdf03-4621-4146-ab68-bbf4a6bb57c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee26451f71e54e0a8a24f30ee6f4eaba', 'request_id': 'req-b98bdf03-4621-4146-ab68-bbf4a6bb57c1'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:23.553620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b98bdf03-4621-4146-ab68-bbf4a6bb57c1 admin admin] The token cache is being invalidate because project ee26451f71e54e0a8a24f30ee6f4eaba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:59:23.553905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b98bdf03-4621-4146-ab68-bbf4a6bb57c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:23.554907 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-b98bdf03-4621-4146-ab68-bbf4a6bb57c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:23.556460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1166/2356] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:59:23 2022] DELETE /identity/v3/projects/ee26451f71e54e0a8a24f30ee6f4eaba => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:23.560422 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe6198-fbaf-4a3e-a8ea-f29f5e38a852 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:23.570439 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe6198-fbaf-4a3e-a8ea-f29f5e38a852 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:23.628013 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98baf936-4bfd-4c65-94f5-3277f39e09fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:23.628647 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98baf936-4bfd-4c65-94f5-3277f39e09fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:23.628926 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98baf936-4bfd-4c65-94f5-3277f39e09fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:23.629209 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98baf936-4bfd-4c65-94f5-3277f39e09fc admin admin] PATH_INFO: `/v3/users/72b904c98d3244d58afa185265e4e749` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:23.630383 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98baf936-4bfd-4c65-94f5-3277f39e09fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=72b904c98d3244d58afa185265e4e749)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:23.632994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98baf936-4bfd-4c65-94f5-3277f39e09fc admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:23.679018 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-98baf936-4bfd-4c65-94f5-3277f39e09fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72b904c98d3244d58afa185265e4e749'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:23.679828 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-98baf936-4bfd-4c65-94f5-3277f39e09fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72b904c98d3244d58afa185265e4e749'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:23.691730 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-98baf936-4bfd-4c65-94f5-3277f39e09fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72b904c98d3244d58afa185265e4e749'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:23.696870 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:23.700292 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-98baf936-4bfd-4c65-94f5-3277f39e09fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72b904c98d3244d58afa185265e4e749'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:23.707923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-98baf936-4bfd-4c65-94f5-3277f39e09fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72b904c98d3244d58afa185265e4e749'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:23.714572 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1191/2357] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:59:23 2022] DELETE /identity/v3/users/72b904c98d3244d58afa185265e4e749 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:59:23.984983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bdf03-4621-4146-ab68-bbf4a6bb57c1 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:23.986449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b98bdf03-4621-4146-ab68-bbf4a6bb57c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:24.000632 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bdf03-4621-4146-ab68-bbf4a6bb57c1 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:24.061811 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e8974-f438-40d4-9c77-895b877bf298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:24.062526 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e8974-f438-40d4-9c77-895b877bf298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:24.062715 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e8974-f438-40d4-9c77-895b877bf298 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:24.062988 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e8974-f438-40d4-9c77-895b877bf298 admin admin] PATH_INFO: `/v3/projects/13c6b43710a240edbf0c0bb4ab9c5c2f` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:24.064120 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e8974-f438-40d4-9c77-895b877bf298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13c6b43710a240edbf0c0bb4ab9c5c2f)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:24.066042 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e8974-f438-40d4-9c77-895b877bf298 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:24.072937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:24.114072 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2f9e8974-f438-40d4-9c77-895b877bf298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13c6b43710a240edbf0c0bb4ab9c5c2f', 'request_id': 'req-2f9e8974-f438-40d4-9c77-895b877bf298'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:24.122230 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2f9e8974-f438-40d4-9c77-895b877bf298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13c6b43710a240edbf0c0bb4ab9c5c2f', 'request_id': 'req-2f9e8974-f438-40d4-9c77-895b877bf298'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:24.128375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2f9e8974-f438-40d4-9c77-895b877bf298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13c6b43710a240edbf0c0bb4ab9c5c2f', 'request_id': 'req-2f9e8974-f438-40d4-9c77-895b877bf298'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:24.139317 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2f9e8974-f438-40d4-9c77-895b877bf298 admin admin] The token cache is being invalidate because project 13c6b43710a240edbf0c0bb4ab9c5c2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:59:24.139605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2f9e8974-f438-40d4-9c77-895b877bf298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:24.140279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2f9e8974-f438-40d4-9c77-895b877bf298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:24.141756 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1167/2358] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:59:23 2022] DELETE /identity/v3/projects/13c6b43710a240edbf0c0bb4ab9c5c2f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:24.146623 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c27a0-30e2-4a69-a9cd-f10a140ff382 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:24.146913 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c27a0-30e2-4a69-a9cd-f10a140ff382 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:24.147194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c27a0-30e2-4a69-a9cd-f10a140ff382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:24.166997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-518c27a0-30e2-4a69-a9cd-f10a140ff382 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:24.168293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-518c27a0-30e2-4a69-a9cd-f10a140ff382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:24.229787 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1192/2359] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:24.237828 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b8ea4-0b65-4fdf-a1dc-835c2400b28f None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:24.238120 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b8ea4-0b65-4fdf-a1dc-835c2400b28f None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:24.238398 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b8ea4-0b65-4fdf-a1dc-835c2400b28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:24.260107 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-ed5b8ea4-0b65-4fdf-a1dc-835c2400b28f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:24.261333 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-ed5b8ea4-0b65-4fdf-a1dc-835c2400b28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:24.269339 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1168/2360] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:24.274463 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c27a0-30e2-4a69-a9cd-f10a140ff382 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:24.284047 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c27a0-30e2-4a69-a9cd-f10a140ff382 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:24.288617 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fd3fa-0b9d-4272-b790-29ed060cedf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:24.289224 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fd3fa-0b9d-4272-b790-29ed060cedf5 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:24.289501 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fd3fa-0b9d-4272-b790-29ed060cedf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:24.289772 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fd3fa-0b9d-4272-b790-29ed060cedf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:24.290407 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fd3fa-0b9d-4272-b790-29ed060cedf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:59:24.290710 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fd3fa-0b9d-4272-b790-29ed060cedf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:24.292325 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fd3fa-0b9d-4272-b790-29ed060cedf5 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:24.303656 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1193/2361] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:59:24.308075 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b8ea4-0b65-4fdf-a1dc-835c2400b28f None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:24.317324 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b8ea4-0b65-4fdf-a1dc-835c2400b28f None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:24.322825 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de08b09-2627-49d0-bb68-cc3f7105ddf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:24.323419 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de08b09-2627-49d0-bb68-cc3f7105ddf2 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:24.323749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de08b09-2627-49d0-bb68-cc3f7105ddf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:24.324054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de08b09-2627-49d0-bb68-cc3f7105ddf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:24.324751 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de08b09-2627-49d0-bb68-cc3f7105ddf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:24.326655 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de08b09-2627-49d0-bb68-cc3f7105ddf2 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:24.354257 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1169/2362] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:59:24 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:59:24.358572 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fd3fa-0b9d-4272-b790-29ed060cedf5 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:24.368391 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fd3fa-0b9d-4272-b790-29ed060cedf5 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:24.424886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77b255-59c1-456a-ba89-b46b494fa5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:24.425418 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77b255-59c1-456a-ba89-b46b494fa5ad admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:24.425693 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77b255-59c1-456a-ba89-b46b494fa5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:24.425961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77b255-59c1-456a-ba89-b46b494fa5ad admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:24.426576 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77b255-59c1-456a-ba89-b46b494fa5ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:24.428519 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77b255-59c1-456a-ba89-b46b494fa5ad admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:24.449659 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1194/2363] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:59:24 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:59:24.453768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de08b09-2627-49d0-bb68-cc3f7105ddf2 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:24.464746 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de08b09-2627-49d0-bb68-cc3f7105ddf2 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:24.469637 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fb625-69ae-4ff8-8b24-ad89dbd00a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:24.470262 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fb625-69ae-4ff8-8b24-ad89dbd00a8e admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:24.470541 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fb625-69ae-4ff8-8b24-ad89dbd00a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:24.470815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fb625-69ae-4ff8-8b24-ad89dbd00a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:24.471449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7fb625-69ae-4ff8-8b24-ad89dbd00a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:24.473076 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7fb625-69ae-4ff8-8b24-ad89dbd00a8e admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:24.483390 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1170/2364] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:59:24.487284 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77b255-59c1-456a-ba89-b46b494fa5ad admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:24.497515 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77b255-59c1-456a-ba89-b46b494fa5ad admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:24.502214 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b917a7-7ea9-49f7-8fa3-1cfe519a468b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:24.502820 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b917a7-7ea9-49f7-8fa3-1cfe519a468b admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:24.503098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b917a7-7ea9-49f7-8fa3-1cfe519a468b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:24.503380 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b917a7-7ea9-49f7-8fa3-1cfe519a468b admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:24.504059 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b917a7-7ea9-49f7-8fa3-1cfe519a468b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:24.505719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b917a7-7ea9-49f7-8fa3-1cfe519a468b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:24.517431 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1195/2365] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:59:24.521634 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fb625-69ae-4ff8-8b24-ad89dbd00a8e admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:24.532322 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fb625-69ae-4ff8-8b24-ad89dbd00a8e admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:24.536901 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82414a8-0c04-4094-8169-808aa1601e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:24.537551 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82414a8-0c04-4094-8169-808aa1601e0d admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:24.537833 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82414a8-0c04-4094-8169-808aa1601e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:24.538111 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82414a8-0c04-4094-8169-808aa1601e0d admin admin] PATH_INFO: `/v3/projects/d4ce78e16c8c4320880cb99276b9f9e5/users/00aab87dfe5a420cbc0d6d036c629623/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:24.552918 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82414a8-0c04-4094-8169-808aa1601e0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4ce78e16c8c4320880cb99276b9f9e5, user_id=00aab87dfe5a420cbc0d6d036c629623, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:24.555833 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82414a8-0c04-4094-8169-808aa1601e0d admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:24.570968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1171/2366] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:59:24 2022] PUT /identity/v3/projects/d4ce78e16c8c4320880cb99276b9f9e5/users/00aab87dfe5a420cbc0d6d036c629623/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:24.581197 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245226ba-f630-4da4-8ee3-e405fb8665d6 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:24.581556 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245226ba-f630-4da4-8ee3-e405fb8665d6 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:24.581831 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245226ba-f630-4da4-8ee3-e405fb8665d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:24.610542 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-245226ba-f630-4da4-8ee3-e405fb8665d6 None None] MFA Rules not processed for user `00aab87dfe5a420cbc0d6d036c629623`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:24.611799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-245226ba-f630-4da4-8ee3-e405fb8665d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:24.655824 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:24.682202 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1196/2367] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:24.686671 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82414a8-0c04-4094-8169-808aa1601e0d admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:24.749090 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82414a8-0c04-4094-8169-808aa1601e0d admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:24.753708 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bea55-285e-4c9f-8cf2-cfd1a813a044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:24.754306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bea55-285e-4c9f-8cf2-cfd1a813a044 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:24.754583 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bea55-285e-4c9f-8cf2-cfd1a813a044 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:24.754851 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bea55-285e-4c9f-8cf2-cfd1a813a044 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:24.755481 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bea55-285e-4c9f-8cf2-cfd1a813a044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:24.757433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bea55-285e-4c9f-8cf2-cfd1a813a044 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:24.785615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1172/2368] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:59:24 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:59:24.789949 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245226ba-f630-4da4-8ee3-e405fb8665d6 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:24.800584 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245226ba-f630-4da4-8ee3-e405fb8665d6 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:24.857469 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb8fef-6711-4578-896b-ff169854db15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:24.858079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb8fef-6711-4578-896b-ff169854db15 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:24.858354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb8fef-6711-4578-896b-ff169854db15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:24.858629 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb8fef-6711-4578-896b-ff169854db15 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:24.859433 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb8fef-6711-4578-896b-ff169854db15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:24.861379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb8fef-6711-4578-896b-ff169854db15 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:24.886052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1197/2369] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:59:24 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:59:24.890036 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bea55-285e-4c9f-8cf2-cfd1a813a044 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:24.900747 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bea55-285e-4c9f-8cf2-cfd1a813a044 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:24.905324 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc469338-3262-4eef-b71b-553b5efcbcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:24.905922 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc469338-3262-4eef-b71b-553b5efcbcc5 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:24.906201 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc469338-3262-4eef-b71b-553b5efcbcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:24.906548 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc469338-3262-4eef-b71b-553b5efcbcc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:24.907313 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc469338-3262-4eef-b71b-553b5efcbcc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:24.908994 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc469338-3262-4eef-b71b-553b5efcbcc5 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:24.919864 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1173/2370] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:59:24.924164 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb8fef-6711-4578-896b-ff169854db15 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:24.935109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb8fef-6711-4578-896b-ff169854db15 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:24.939634 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977caefa-3c5b-4735-8cb5-d7ccdc94db80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:24.940256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977caefa-3c5b-4735-8cb5-d7ccdc94db80 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:24.940534 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977caefa-3c5b-4735-8cb5-d7ccdc94db80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:24.940807 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977caefa-3c5b-4735-8cb5-d7ccdc94db80 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:24.941440 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977caefa-3c5b-4735-8cb5-d7ccdc94db80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:24.943037 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977caefa-3c5b-4735-8cb5-d7ccdc94db80 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:24.953997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1198/2371] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:59:24.958193 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc469338-3262-4eef-b71b-553b5efcbcc5 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:24.964821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:24.968853 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc469338-3262-4eef-b71b-553b5efcbcc5 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:24.974609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f219a-3b78-4c5b-b2ce-fb0dc7f21374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:24.975420 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f219a-3b78-4c5b-b2ce-fb0dc7f21374 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:24.975913 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f219a-3b78-4c5b-b2ce-fb0dc7f21374 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:24.977957 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f219a-3b78-4c5b-b2ce-fb0dc7f21374 admin admin] PATH_INFO: `/v3/projects/33245fb21a314e35b5294dfa51961ce5/users/22051d3c7c6843088a054157f5a76bd3/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:24.991727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f219a-3b78-4c5b-b2ce-fb0dc7f21374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33245fb21a314e35b5294dfa51961ce5, user_id=22051d3c7c6843088a054157f5a76bd3, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:24.995064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f219a-3b78-4c5b-b2ce-fb0dc7f21374 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:25.008135 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1174/2372] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:59:24 2022] PUT /identity/v3/projects/33245fb21a314e35b5294dfa51961ce5/users/22051d3c7c6843088a054157f5a76bd3/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:25.019379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fc74a-f63e-460b-b613-e923771974cb None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:25.019667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fc74a-f63e-460b-b613-e923771974cb None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:25.019943 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fc74a-f63e-460b-b613-e923771974cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:25.047998 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-401fc74a-f63e-460b-b613-e923771974cb None None] MFA Rules not processed for user `22051d3c7c6843088a054157f5a76bd3`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:25.049256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-401fc74a-f63e-460b-b613-e923771974cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:25.112959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1199/2373] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:25.118163 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f219a-3b78-4c5b-b2ce-fb0dc7f21374 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:25.178194 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f219a-3b78-4c5b-b2ce-fb0dc7f21374 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:25.183771 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fb5de-8ee8-4abb-baf4-fef6e54b6268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:25.184392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fb5de-8ee8-4abb-baf4-fef6e54b6268 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:25.184672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fb5de-8ee8-4abb-baf4-fef6e54b6268 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:25.184944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fb5de-8ee8-4abb-baf4-fef6e54b6268 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:25.185632 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466fb5de-8ee8-4abb-baf4-fef6e54b6268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:25.187530 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466fb5de-8ee8-4abb-baf4-fef6e54b6268 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:25.213842 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1175/2374] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:59:25 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:59:25.218413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fc74a-f63e-460b-b613-e923771974cb None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:25.229950 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fc74a-f63e-460b-b613-e923771974cb None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:25.276344 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:25.287099 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c8c99-f1de-4f5a-b7c7-aef4e04230de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:25.287709 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c8c99-f1de-4f5a-b7c7-aef4e04230de admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:25.287989 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c8c99-f1de-4f5a-b7c7-aef4e04230de admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:25.288293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c8c99-f1de-4f5a-b7c7-aef4e04230de admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:25.289802 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c8c99-f1de-4f5a-b7c7-aef4e04230de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:25.291719 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c8c99-f1de-4f5a-b7c7-aef4e04230de admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:25.313385 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1200/2375] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:59:25 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:59:25.317478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fb5de-8ee8-4abb-baf4-fef6e54b6268 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:25.327658 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fb5de-8ee8-4abb-baf4-fef6e54b6268 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:25.332203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77bb37-afcc-4146-9f22-c9672c839266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:25.332790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77bb37-afcc-4146-9f22-c9672c839266 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:25.333174 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77bb37-afcc-4146-9f22-c9672c839266 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:25.333454 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77bb37-afcc-4146-9f22-c9672c839266 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:25.334079 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77bb37-afcc-4146-9f22-c9672c839266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:25.336195 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77bb37-afcc-4146-9f22-c9672c839266 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:25.346927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1176/2376] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:59:25.350851 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c8c99-f1de-4f5a-b7c7-aef4e04230de admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:25.361305 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c8c99-f1de-4f5a-b7c7-aef4e04230de admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:25.365812 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768ca91-0e1a-45e0-89b8-bf8a1531bb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:25.366452 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768ca91-0e1a-45e0-89b8-bf8a1531bb76 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:25.366728 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768ca91-0e1a-45e0-89b8-bf8a1531bb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:25.367027 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768ca91-0e1a-45e0-89b8-bf8a1531bb76 admin admin] PATH_INFO: `/v3/projects/6338f0b903524a9db2846f051b597036/users/ee7de78b70de4967bb5733c0a4df5180/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:25.381637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768ca91-0e1a-45e0-89b8-bf8a1531bb76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6338f0b903524a9db2846f051b597036, user_id=ee7de78b70de4967bb5733c0a4df5180, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:25.384313 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8768ca91-0e1a-45e0-89b8-bf8a1531bb76 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:25.396759 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1201/2377] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:59:25 2022] PUT /identity/v3/projects/6338f0b903524a9db2846f051b597036/users/ee7de78b70de4967bb5733c0a4df5180/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:59:25.401292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77bb37-afcc-4146-9f22-c9672c839266 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:25.411408 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77bb37-afcc-4146-9f22-c9672c839266 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:25.467971 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21a12a-6c45-4e1f-8324-575cdfca2c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:25.468525 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21a12a-6c45-4e1f-8324-575cdfca2c40 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:25.468806 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21a12a-6c45-4e1f-8324-575cdfca2c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:25.469078 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21a12a-6c45-4e1f-8324-575cdfca2c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:25.469714 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e21a12a-6c45-4e1f-8324-575cdfca2c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:25.471318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e21a12a-6c45-4e1f-8324-575cdfca2c40 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:25.481620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1177/2378] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:59:25.485791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768ca91-0e1a-45e0-89b8-bf8a1531bb76 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:25.496168 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768ca91-0e1a-45e0-89b8-bf8a1531bb76 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:25.500834 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c6dad-ff3a-409c-8000-23f93cc95853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:25.501488 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c6dad-ff3a-409c-8000-23f93cc95853 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:25.501771 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c6dad-ff3a-409c-8000-23f93cc95853 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:25.502041 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c6dad-ff3a-409c-8000-23f93cc95853 admin admin] PATH_INFO: `/v3/projects/6338f0b903524a9db2846f051b597036/users/ee7de78b70de4967bb5733c0a4df5180/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:25.503944 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c6dad-ff3a-409c-8000-23f93cc95853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6338f0b903524a9db2846f051b597036, user_id=ee7de78b70de4967bb5733c0a4df5180, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:25.506613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c6dad-ff3a-409c-8000-23f93cc95853 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:25.517921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1202/2379] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:59:25 2022] PUT /identity/v3/projects/6338f0b903524a9db2846f051b597036/users/ee7de78b70de4967bb5733c0a4df5180/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:25.528497 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac154012-0e28-4ef9-8dd2-040bb87d7cbe None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:25.528785 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac154012-0e28-4ef9-8dd2-040bb87d7cbe None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:25.529061 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac154012-0e28-4ef9-8dd2-040bb87d7cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:25.557872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-ac154012-0e28-4ef9-8dd2-040bb87d7cbe None None] MFA Rules not processed for user `ee7de78b70de4967bb5733c0a4df5180`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:25.559088 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-ac154012-0e28-4ef9-8dd2-040bb87d7cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:25.599776 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:25.629610 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1178/2380] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:25.641739 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c6dad-ff3a-409c-8000-23f93cc95853 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:25.643456 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f77c6dad-ff3a-409c-8000-23f93cc95853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:25.701015 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c6dad-ff3a-409c-8000-23f93cc95853 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:25.704613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94b09e-659b-45ab-8ef1-81f20b4713d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:25.705232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94b09e-659b-45ab-8ef1-81f20b4713d6 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:25.705508 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94b09e-659b-45ab-8ef1-81f20b4713d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:25.705778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94b09e-659b-45ab-8ef1-81f20b4713d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:25.762913 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94b09e-659b-45ab-8ef1-81f20b4713d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:25.764321 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94b09e-659b-45ab-8ef1-81f20b4713d6 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:25.772889 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1203/2381] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:59:27.285180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac154012-0e28-4ef9-8dd2-040bb87d7cbe None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:27.286584 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-ac154012-0e28-4ef9-8dd2-040bb87d7cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:27.298409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac154012-0e28-4ef9-8dd2-040bb87d7cbe None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:27.301892 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af95e97-bc22-4e70-afd0-3f5db09c2ad1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:27.302486 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af95e97-bc22-4e70-afd0-3f5db09c2ad1 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:27.302762 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af95e97-bc22-4e70-afd0-3f5db09c2ad1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:27.303030 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af95e97-bc22-4e70-afd0-3f5db09c2ad1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:27.360088 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af95e97-bc22-4e70-afd0-3f5db09c2ad1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:27.361554 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af95e97-bc22-4e70-afd0-3f5db09c2ad1 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:27.370442 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1179/2382] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:59:27.897125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94b09e-659b-45ab-8ef1-81f20b4713d6 service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:27.906390 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94b09e-659b-45ab-8ef1-81f20b4713d6 service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:27.909973 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2a477-da4b-4c6e-80d5-dca46b5a2630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:27.910566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2a477-da4b-4c6e-80d5-dca46b5a2630 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:27.910847 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2a477-da4b-4c6e-80d5-dca46b5a2630 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:27.911121 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2a477-da4b-4c6e-80d5-dca46b5a2630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:27.920447 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc2a477-da4b-4c6e-80d5-dca46b5a2630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:27.921867 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc2a477-da4b-4c6e-80d5-dca46b5a2630 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:27.931319 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1204/2383] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:59:31.801004 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af95e97-bc22-4e70-afd0-3f5db09c2ad1 service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:31.862218 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af95e97-bc22-4e70-afd0-3f5db09c2ad1 service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:31.866781 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c62807-1794-4d81-b7c7-af5f0a58b1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:31.867401 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c62807-1794-4d81-b7c7-af5f0a58b1fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:31.867675 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c62807-1794-4d81-b7c7-af5f0a58b1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:31.867943 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c62807-1794-4d81-b7c7-af5f0a58b1fd admin admin] PATH_INFO: `/v3/users/00aab87dfe5a420cbc0d6d036c629623` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:31.869806 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c62807-1794-4d81-b7c7-af5f0a58b1fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=00aab87dfe5a420cbc0d6d036c629623)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:31.871736 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c62807-1794-4d81-b7c7-af5f0a58b1fd admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:31.919491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-67c62807-1794-4d81-b7c7-af5f0a58b1fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00aab87dfe5a420cbc0d6d036c629623'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:31.920283 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-67c62807-1794-4d81-b7c7-af5f0a58b1fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00aab87dfe5a420cbc0d6d036c629623'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:31.932528 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-67c62807-1794-4d81-b7c7-af5f0a58b1fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00aab87dfe5a420cbc0d6d036c629623'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:31.939272 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-67c62807-1794-4d81-b7c7-af5f0a58b1fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00aab87dfe5a420cbc0d6d036c629623'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:31.939910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-67c62807-1794-4d81-b7c7-af5f0a58b1fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00aab87dfe5a420cbc0d6d036c629623'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:31.942422 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:31.953570 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1180/2384] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:59:31 2022] DELETE /identity/v3/users/00aab87dfe5a420cbc0d6d036c629623 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:31.958104 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfa6e7-2932-4e51-a546-717987443377 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:31.958591 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfa6e7-2932-4e51-a546-717987443377 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:31.958591 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfa6e7-2932-4e51-a546-717987443377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:31.978401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-54cfa6e7-2932-4e51-a546-717987443377 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:31.979592 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-54cfa6e7-2932-4e51-a546-717987443377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:32.018825 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:32.038322 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1205/2385] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:32.048774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c62807-1794-4d81-b7c7-af5f0a58b1fd admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:32.050221 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-67c62807-1794-4d81-b7c7-af5f0a58b1fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:32.107633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c62807-1794-4d81-b7c7-af5f0a58b1fd admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:32.111362 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c74b31-71be-4662-a306-01f7af7a0555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:32.111968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c74b31-71be-4662-a306-01f7af7a0555 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:32.112275 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c74b31-71be-4662-a306-01f7af7a0555 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:32.112548 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c74b31-71be-4662-a306-01f7af7a0555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:32.121537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c74b31-71be-4662-a306-01f7af7a0555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:32.122963 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c74b31-71be-4662-a306-01f7af7a0555 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:32.133987 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1181/2386] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 00:59:32.404453 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfa6e7-2932-4e51-a546-717987443377 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:32.405962 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-54cfa6e7-2932-4e51-a546-717987443377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:32.420783 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfa6e7-2932-4e51-a546-717987443377 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:32.425547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf63f91-b1e4-422f-9c5b-66ba1e124118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:32.426184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf63f91-b1e4-422f-9c5b-66ba1e124118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:32.426465 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf63f91-b1e4-422f-9c5b-66ba1e124118 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:32.426740 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf63f91-b1e4-422f-9c5b-66ba1e124118 admin admin] PATH_INFO: `/v3/projects/d4ce78e16c8c4320880cb99276b9f9e5` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:32.427865 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf63f91-b1e4-422f-9c5b-66ba1e124118 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4ce78e16c8c4320880cb99276b9f9e5)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:32.430859 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf63f91-b1e4-422f-9c5b-66ba1e124118 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:32.479632 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ecf63f91-b1e4-422f-9c5b-66ba1e124118 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4ce78e16c8c4320880cb99276b9f9e5', 'request_id': 'req-ecf63f91-b1e4-422f-9c5b-66ba1e124118'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:32.487048 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ecf63f91-b1e4-422f-9c5b-66ba1e124118 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4ce78e16c8c4320880cb99276b9f9e5', 'request_id': 'req-ecf63f91-b1e4-422f-9c5b-66ba1e124118'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:32.492657 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ecf63f91-b1e4-422f-9c5b-66ba1e124118 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4ce78e16c8c4320880cb99276b9f9e5', 'request_id': 'req-ecf63f91-b1e4-422f-9c5b-66ba1e124118'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:32.503998 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ecf63f91-b1e4-422f-9c5b-66ba1e124118 admin admin] The token cache is being invalidate because project d4ce78e16c8c4320880cb99276b9f9e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:59:32.504320 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ecf63f91-b1e4-422f-9c5b-66ba1e124118 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:32.504945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ecf63f91-b1e4-422f-9c5b-66ba1e124118 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:32.506454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1206/2387] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:59:32 2022] DELETE /identity/v3/projects/d4ce78e16c8c4320880cb99276b9f9e5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:32.511016 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c74b31-71be-4662-a306-01f7af7a0555 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:32.520695 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c74b31-71be-4662-a306-01f7af7a0555 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:32.577737 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a713c-9383-443b-8f31-cdf7e49f0424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:32.578361 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a713c-9383-443b-8f31-cdf7e49f0424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:32.578643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a713c-9383-443b-8f31-cdf7e49f0424 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:32.578917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a713c-9383-443b-8f31-cdf7e49f0424 admin admin] PATH_INFO: `/v3/users/22051d3c7c6843088a054157f5a76bd3` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:32.580026 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a713c-9383-443b-8f31-cdf7e49f0424 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22051d3c7c6843088a054157f5a76bd3)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:32.581983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a713c-9383-443b-8f31-cdf7e49f0424 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:32.627080 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-565a713c-9383-443b-8f31-cdf7e49f0424 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22051d3c7c6843088a054157f5a76bd3'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:32.627723 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-565a713c-9383-443b-8f31-cdf7e49f0424 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22051d3c7c6843088a054157f5a76bd3'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:32.641742 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-565a713c-9383-443b-8f31-cdf7e49f0424 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22051d3c7c6843088a054157f5a76bd3'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:32.648738 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-565a713c-9383-443b-8f31-cdf7e49f0424 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22051d3c7c6843088a054157f5a76bd3'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:32.650388 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-565a713c-9383-443b-8f31-cdf7e49f0424 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22051d3c7c6843088a054157f5a76bd3'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:32.662890 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1182/2388] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:59:32 2022] DELETE /identity/v3/users/22051d3c7c6843088a054157f5a76bd3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:32.938154 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf63f91-b1e4-422f-9c5b-66ba1e124118 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:32.939613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ecf63f91-b1e4-422f-9c5b-66ba1e124118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:32.953210 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf63f91-b1e4-422f-9c5b-66ba1e124118 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:33.011560 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896001e-78a7-40de-927b-95f1ea50a4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:33.012240 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896001e-78a7-40de-927b-95f1ea50a4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:33.012442 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896001e-78a7-40de-927b-95f1ea50a4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:33.012716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896001e-78a7-40de-927b-95f1ea50a4ba admin admin] PATH_INFO: `/v3/projects/33245fb21a314e35b5294dfa51961ce5` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:33.013834 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896001e-78a7-40de-927b-95f1ea50a4ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=33245fb21a314e35b5294dfa51961ce5)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:33.015793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896001e-78a7-40de-927b-95f1ea50a4ba admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:33.061806 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7896001e-78a7-40de-927b-95f1ea50a4ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33245fb21a314e35b5294dfa51961ce5', 'request_id': 'req-7896001e-78a7-40de-927b-95f1ea50a4ba'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:33.068776 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7896001e-78a7-40de-927b-95f1ea50a4ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33245fb21a314e35b5294dfa51961ce5', 'request_id': 'req-7896001e-78a7-40de-927b-95f1ea50a4ba'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:33.074401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7896001e-78a7-40de-927b-95f1ea50a4ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33245fb21a314e35b5294dfa51961ce5', 'request_id': 'req-7896001e-78a7-40de-927b-95f1ea50a4ba'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:33.081390 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:33.086717 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7896001e-78a7-40de-927b-95f1ea50a4ba admin admin] The token cache is being invalidate because project 33245fb21a314e35b5294dfa51961ce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:59:33.088190 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7896001e-78a7-40de-927b-95f1ea50a4ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:33.088532 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7896001e-78a7-40de-927b-95f1ea50a4ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:33.090392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1207/2389] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:59:32 2022] DELETE /identity/v3/projects/33245fb21a314e35b5294dfa51961ce5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:33.094641 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a713c-9383-443b-8f31-cdf7e49f0424 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:33.096068 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-565a713c-9383-443b-8f31-cdf7e49f0424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:33.109346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a713c-9383-443b-8f31-cdf7e49f0424 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:33.121075 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:33.168035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9bf5f-f136-4dae-8918-748a02a4611f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:33.168701 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9bf5f-f136-4dae-8918-748a02a4611f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:33.168983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9bf5f-f136-4dae-8918-748a02a4611f admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:33.169258 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9bf5f-f136-4dae-8918-748a02a4611f admin admin] PATH_INFO: `/v3/users/ee7de78b70de4967bb5733c0a4df5180` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:33.170363 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9bf5f-f136-4dae-8918-748a02a4611f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee7de78b70de4967bb5733c0a4df5180)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:33.172352 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9bf5f-f136-4dae-8918-748a02a4611f admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:33.217443 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1ec9bf5f-f136-4dae-8918-748a02a4611f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee7de78b70de4967bb5733c0a4df5180'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:33.218349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1ec9bf5f-f136-4dae-8918-748a02a4611f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee7de78b70de4967bb5733c0a4df5180'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:33.230404 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1ec9bf5f-f136-4dae-8918-748a02a4611f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee7de78b70de4967bb5733c0a4df5180'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:33.237231 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1ec9bf5f-f136-4dae-8918-748a02a4611f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee7de78b70de4967bb5733c0a4df5180'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:33.237886 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1ec9bf5f-f136-4dae-8918-748a02a4611f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee7de78b70de4967bb5733c0a4df5180'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:33.250934 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1183/2390] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 00:59:33 2022] DELETE /identity/v3/users/ee7de78b70de4967bb5733c0a4df5180 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:33.526264 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896001e-78a7-40de-927b-95f1ea50a4ba admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:33.527729 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-7896001e-78a7-40de-927b-95f1ea50a4ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:33.541006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896001e-78a7-40de-927b-95f1ea50a4ba admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:33.599301 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c588de-103d-4160-b20e-d2d0a32610b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:33.599936 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c588de-103d-4160-b20e-d2d0a32610b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:33.600255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c588de-103d-4160-b20e-d2d0a32610b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:33.600533 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c588de-103d-4160-b20e-d2d0a32610b1 admin admin] PATH_INFO: `/v3/projects/6338f0b903524a9db2846f051b597036` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:33.601649 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c588de-103d-4160-b20e-d2d0a32610b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6338f0b903524a9db2846f051b597036)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:33.603699 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c588de-103d-4160-b20e-d2d0a32610b1 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:33.649628 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d3c588de-103d-4160-b20e-d2d0a32610b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6338f0b903524a9db2846f051b597036', 'request_id': 'req-d3c588de-103d-4160-b20e-d2d0a32610b1'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:33.656656 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d3c588de-103d-4160-b20e-d2d0a32610b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6338f0b903524a9db2846f051b597036', 'request_id': 'req-d3c588de-103d-4160-b20e-d2d0a32610b1'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:33.662115 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d3c588de-103d-4160-b20e-d2d0a32610b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6338f0b903524a9db2846f051b597036', 'request_id': 'req-d3c588de-103d-4160-b20e-d2d0a32610b1'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:33.672925 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d3c588de-103d-4160-b20e-d2d0a32610b1 admin admin] The token cache is being invalidate because project 6338f0b903524a9db2846f051b597036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 00:59:33.673215 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d3c588de-103d-4160-b20e-d2d0a32610b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:33.673842 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d3c588de-103d-4160-b20e-d2d0a32610b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 00:59:33.675302 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1208/2391] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 00:59:33 2022] DELETE /identity/v3/projects/6338f0b903524a9db2846f051b597036 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 00:59:33.680322 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81698253-6245-42e5-8b73-10b3723a1b6f None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:33.681093 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81698253-6245-42e5-8b73-10b3723a1b6f None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:33.681380 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81698253-6245-42e5-8b73-10b3723a1b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:33.701374 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-81698253-6245-42e5-8b73-10b3723a1b6f None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:33.702584 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-81698253-6245-42e5-8b73-10b3723a1b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:33.762677 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1184/2392] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:33.770491 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76ee06-01d6-4015-884a-8f87cced7e05 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:33.770716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76ee06-01d6-4015-884a-8f87cced7e05 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:33.770992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76ee06-01d6-4015-884a-8f87cced7e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:33.791495 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-ba76ee06-01d6-4015-884a-8f87cced7e05 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:33.792716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ba76ee06-01d6-4015-884a-8f87cced7e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:33.800812 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1209/2393] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:33.804951 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81698253-6245-42e5-8b73-10b3723a1b6f None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:33.815340 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81698253-6245-42e5-8b73-10b3723a1b6f None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:33.819928 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be112e4b-40b7-47f2-ba13-c5103811efe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:33.820555 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be112e4b-40b7-47f2-ba13-c5103811efe0 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:33.820837 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be112e4b-40b7-47f2-ba13-c5103811efe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:33.821108 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be112e4b-40b7-47f2-ba13-c5103811efe0 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:33.822634 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be112e4b-40b7-47f2-ba13-c5103811efe0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 00:59:33.822943 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be112e4b-40b7-47f2-ba13-c5103811efe0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:33.824584 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be112e4b-40b7-47f2-ba13-c5103811efe0 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:33.835965 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1185/2394] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 00:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 00:59:33.841792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76ee06-01d6-4015-884a-8f87cced7e05 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:33.851256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76ee06-01d6-4015-884a-8f87cced7e05 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:33.856151 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca54003-5cd9-4791-a504-9a7b1bc230a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:33.856746 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca54003-5cd9-4791-a504-9a7b1bc230a9 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:33.857027 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca54003-5cd9-4791-a504-9a7b1bc230a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:33.857298 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca54003-5cd9-4791-a504-9a7b1bc230a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:33.857926 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca54003-5cd9-4791-a504-9a7b1bc230a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:33.859862 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca54003-5cd9-4791-a504-9a7b1bc230a9 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:33.888282 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1210/2395] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 00:59:33 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:59:33.892402 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be112e4b-40b7-47f2-ba13-c5103811efe0 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:33.902702 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be112e4b-40b7-47f2-ba13-c5103811efe0 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:33.923922 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:33.963519 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e85d5-d8fa-452e-b65f-198635a2fae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:33.964143 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e85d5-d8fa-452e-b65f-198635a2fae2 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:33.964432 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e85d5-d8fa-452e-b65f-198635a2fae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:33.964705 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e85d5-d8fa-452e-b65f-198635a2fae2 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:33.965324 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e85d5-d8fa-452e-b65f-198635a2fae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:33.967253 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e85d5-d8fa-452e-b65f-198635a2fae2 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:33.988022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1186/2396] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 00:59:33 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 00:59:33.992280 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca54003-5cd9-4791-a504-9a7b1bc230a9 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:34.003242 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca54003-5cd9-4791-a504-9a7b1bc230a9 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:34.007884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0daaae-00c5-445c-b748-953f88f93784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:34.008515 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0daaae-00c5-445c-b748-953f88f93784 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:34.008795 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0daaae-00c5-445c-b748-953f88f93784 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:34.009327 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0daaae-00c5-445c-b748-953f88f93784 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:34.009854 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0daaae-00c5-445c-b748-953f88f93784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:34.011442 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0daaae-00c5-445c-b748-953f88f93784 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:34.021962 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1211/2397] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 00:59:34.026189 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e85d5-d8fa-452e-b65f-198635a2fae2 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:34.038242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e85d5-d8fa-452e-b65f-198635a2fae2 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:34.043662 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875a0cd-31fd-4304-ba82-81052529d8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:34.044286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875a0cd-31fd-4304-ba82-81052529d8ea admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:34.044566 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875a0cd-31fd-4304-ba82-81052529d8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:34.044837 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875a0cd-31fd-4304-ba82-81052529d8ea admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:34.045464 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875a0cd-31fd-4304-ba82-81052529d8ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:34.047056 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875a0cd-31fd-4304-ba82-81052529d8ea admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:34.057479 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1187/2398] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 00:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 00:59:34.061599 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0daaae-00c5-445c-b748-953f88f93784 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:34.071348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0daaae-00c5-445c-b748-953f88f93784 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:34.076030 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fafe9-bde4-426f-a172-5948e4f6ce3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:34.076691 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fafe9-bde4-426f-a172-5948e4f6ce3e admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:34.076965 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fafe9-bde4-426f-a172-5948e4f6ce3e admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:34.077240 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fafe9-bde4-426f-a172-5948e4f6ce3e admin admin] PATH_INFO: `/v3/projects/4c5fc6ec2b1342fda46cc1499dce4d87/users/1d68778d06544601a9452be0cba82b5d/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:34.091372 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077fafe9-bde4-426f-a172-5948e4f6ce3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5fc6ec2b1342fda46cc1499dce4d87, user_id=1d68778d06544601a9452be0cba82b5d, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:34.094285 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077fafe9-bde4-426f-a172-5948e4f6ce3e admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:34.107489 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1212/2399] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 00:59:34 2022] PUT /identity/v3/projects/4c5fc6ec2b1342fda46cc1499dce4d87/users/1d68778d06544601a9452be0cba82b5d/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 00:59:34.171889 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5a28d-062f-4cd3-bdc7-d1081b41ab18 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:34.172547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5a28d-062f-4cd3-bdc7-d1081b41ab18 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:34.172547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5a28d-062f-4cd3-bdc7-d1081b41ab18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:34.202419 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-77e5a28d-062f-4cd3-bdc7-d1081b41ab18 None None] MFA Rules not processed for user `1d68778d06544601a9452be0cba82b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 00:59:34.203630 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-77e5a28d-062f-4cd3-bdc7-d1081b41ab18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:34.267597 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1188/2400] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 00:59:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 00:59:34.279195 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fafe9-bde4-426f-a172-5948e4f6ce3e admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 00:59:34.280932 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-077fafe9-bde4-426f-a172-5948e4f6ce3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 00:59:34.310079 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:34.362692 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fafe9-bde4-426f-a172-5948e4f6ce3e admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 00:59:34.366357 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1eba22-6a77-4210-9638-7cbf205b1423 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cf9451f2ef7478589b4a4278cb03252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 00:59:34.366959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1eba22-6a77-4210-9638-7cbf205b1423 service glance] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 00:59:34.367240 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1eba22-6a77-4210-9638-7cbf205b1423 service glance] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 00:59:34.367515 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1eba22-6a77-4210-9638-7cbf205b1423 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 00:59:34.375735 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1eba22-6a77-4210-9638-7cbf205b1423 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 00:59:34.377191 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1eba22-6a77-4210-9638-7cbf205b1423 service glance] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 00:59:34.385907 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1213/2401] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 00:59:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 00:59:44.264740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=56,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 00:59:44.381866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=24 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:08.476736 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5a28d-062f-4cd3-bdc7-d1081b41ab18 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:08.537770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5a28d-062f-4cd3-bdc7-d1081b41ab18 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:08.542296 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2ea38-783b-49cd-87ae-7f7dc1dcccf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:08.542923 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2ea38-783b-49cd-87ae-7f7dc1dcccf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:08.543206 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2ea38-783b-49cd-87ae-7f7dc1dcccf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:08.543483 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2ea38-783b-49cd-87ae-7f7dc1dcccf5 admin admin] PATH_INFO: `/v3/users/1d68778d06544601a9452be0cba82b5d` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:08.544629 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2ea38-783b-49cd-87ae-7f7dc1dcccf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d68778d06544601a9452be0cba82b5d)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:08.547201 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2ea38-783b-49cd-87ae-7f7dc1dcccf5 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:08.591552 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-99e2ea38-783b-49cd-87ae-7f7dc1dcccf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d68778d06544601a9452be0cba82b5d'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:00:08.592237 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-99e2ea38-783b-49cd-87ae-7f7dc1dcccf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d68778d06544601a9452be0cba82b5d'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:00:08.605219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-99e2ea38-783b-49cd-87ae-7f7dc1dcccf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d68778d06544601a9452be0cba82b5d'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:00:08.612734 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-99e2ea38-783b-49cd-87ae-7f7dc1dcccf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d68778d06544601a9452be0cba82b5d'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:00:08.613376 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-99e2ea38-783b-49cd-87ae-7f7dc1dcccf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d68778d06544601a9452be0cba82b5d'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:00:08.625531 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1189/2402] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 01:00:08 2022] DELETE /identity/v3/users/1d68778d06544601a9452be0cba82b5d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:00:08.630973 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1aa5dc-0ff2-44c6-bf27-69231655b2de None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:08.631454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1aa5dc-0ff2-44c6-bf27-69231655b2de None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:08.631552 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1aa5dc-0ff2-44c6-bf27-69231655b2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:08.650865 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-fe1aa5dc-0ff2-44c6-bf27-69231655b2de None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:00:08.652180 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-fe1aa5dc-0ff2-44c6-bf27-69231655b2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:08.712136 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1214/2403] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:00:08.722653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2ea38-783b-49cd-87ae-7f7dc1dcccf5 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:08.724100 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-99e2ea38-783b-49cd-87ae-7f7dc1dcccf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:08.780273 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2ea38-783b-49cd-87ae-7f7dc1dcccf5 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:08.784638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb3a44-2664-409f-9c9b-c0e512bb667a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:08.785237 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb3a44-2664-409f-9c9b-c0e512bb667a service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:08.785516 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb3a44-2664-409f-9c9b-c0e512bb667a service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:08.785790 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb3a44-2664-409f-9c9b-c0e512bb667a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:08.794721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb3a44-2664-409f-9c9b-c0e512bb667a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:08.796217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb3a44-2664-409f-9c9b-c0e512bb667a service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:08.805953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1190/2404] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:00:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:00:09.072724 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1aa5dc-0ff2-44c6-bf27-69231655b2de None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:09.074131 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-fe1aa5dc-0ff2-44c6-bf27-69231655b2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:09.087669 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1aa5dc-0ff2-44c6-bf27-69231655b2de None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:09.092621 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3c91f-0aae-46fa-b42d-13023537ce76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:09.093316 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3c91f-0aae-46fa-b42d-13023537ce76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:09.094193 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3c91f-0aae-46fa-b42d-13023537ce76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:09.094537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3c91f-0aae-46fa-b42d-13023537ce76 admin admin] PATH_INFO: `/v3/projects/4c5fc6ec2b1342fda46cc1499dce4d87` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:09.095929 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc3c91f-0aae-46fa-b42d-13023537ce76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c5fc6ec2b1342fda46cc1499dce4d87)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:09.097949 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc3c91f-0aae-46fa-b42d-13023537ce76 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:09.148067 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-acc3c91f-0aae-46fa-b42d-13023537ce76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c5fc6ec2b1342fda46cc1499dce4d87', 'request_id': 'req-acc3c91f-0aae-46fa-b42d-13023537ce76'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:00:09.155397 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-acc3c91f-0aae-46fa-b42d-13023537ce76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c5fc6ec2b1342fda46cc1499dce4d87', 'request_id': 'req-acc3c91f-0aae-46fa-b42d-13023537ce76'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:00:09.161576 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-acc3c91f-0aae-46fa-b42d-13023537ce76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c5fc6ec2b1342fda46cc1499dce4d87', 'request_id': 'req-acc3c91f-0aae-46fa-b42d-13023537ce76'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:00:09.172830 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-acc3c91f-0aae-46fa-b42d-13023537ce76 admin admin] The token cache is being invalidate because project 4c5fc6ec2b1342fda46cc1499dce4d87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:00:09.173124 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-acc3c91f-0aae-46fa-b42d-13023537ce76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:00:09.173770 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-acc3c91f-0aae-46fa-b42d-13023537ce76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:00:09.175283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1215/2405] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 01:00:09 2022] DELETE /identity/v3/projects/4c5fc6ec2b1342fda46cc1499dce4d87 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:00:09.181460 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7a450-124e-427d-9e52-6e000f544bf8 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:09.181777 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7a450-124e-427d-9e52-6e000f544bf8 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:09.182056 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7a450-124e-427d-9e52-6e000f544bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:09.201833 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-73e7a450-124e-427d-9e52-6e000f544bf8 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:00:09.203715 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-73e7a450-124e-427d-9e52-6e000f544bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:09.220634 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:09.262857 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1191/2406] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:00:09.271426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04c1a1-8105-40fc-af82-b26bfddf96e7 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:09.271711 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04c1a1-8105-40fc-af82-b26bfddf96e7 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:09.271993 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04c1a1-8105-40fc-af82-b26bfddf96e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:09.291755 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-bb04c1a1-8105-40fc-af82-b26bfddf96e7 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:00:09.293001 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-bb04c1a1-8105-40fc-af82-b26bfddf96e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:09.301093 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1216/2407] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:00:09.305187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7a450-124e-427d-9e52-6e000f544bf8 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:09.315295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7a450-124e-427d-9e52-6e000f544bf8 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:09.319809 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c039362-3b51-47a8-b47a-1f28dbf94d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:09.320443 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c039362-3b51-47a8-b47a-1f28dbf94d32 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:09.320722 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c039362-3b51-47a8-b47a-1f28dbf94d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:09.320997 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c039362-3b51-47a8-b47a-1f28dbf94d32 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:09.321636 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c039362-3b51-47a8-b47a-1f28dbf94d32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:00:09.321939 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c039362-3b51-47a8-b47a-1f28dbf94d32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:09.323540 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c039362-3b51-47a8-b47a-1f28dbf94d32 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:09.335627 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1192/2408] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 01:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:00:09.339661 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04c1a1-8105-40fc-af82-b26bfddf96e7 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:09.348972 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04c1a1-8105-40fc-af82-b26bfddf96e7 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:09.353505 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae665d-81ea-428e-89da-bcea1c4ef1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:09.354092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae665d-81ea-428e-89da-bcea1c4ef1b7 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:09.354370 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae665d-81ea-428e-89da-bcea1c4ef1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:09.354642 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae665d-81ea-428e-89da-bcea1c4ef1b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:09.355276 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae665d-81ea-428e-89da-bcea1c4ef1b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:09.357196 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae665d-81ea-428e-89da-bcea1c4ef1b7 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:09.385131 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1217/2409] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 01:00:09 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:00:09.389164 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c039362-3b51-47a8-b47a-1f28dbf94d32 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:09.399723 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c039362-3b51-47a8-b47a-1f28dbf94d32 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:09.454380 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b0051-d7e0-40d7-a19e-869f16cf2139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:09.455028 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b0051-d7e0-40d7-a19e-869f16cf2139 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:09.455398 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b0051-d7e0-40d7-a19e-869f16cf2139 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:09.455678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b0051-d7e0-40d7-a19e-869f16cf2139 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:09.456339 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b0051-d7e0-40d7-a19e-869f16cf2139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:09.458271 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b0051-d7e0-40d7-a19e-869f16cf2139 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:09.479454 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1193/2410] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 01:00:09 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:00:09.483377 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae665d-81ea-428e-89da-bcea1c4ef1b7 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:09.494177 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae665d-81ea-428e-89da-bcea1c4ef1b7 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:09.498808 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae87dd-516b-4818-8fd1-c5f99f000a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:09.499400 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae87dd-516b-4818-8fd1-c5f99f000a90 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:09.500274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae87dd-516b-4818-8fd1-c5f99f000a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:09.500553 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae87dd-516b-4818-8fd1-c5f99f000a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:09.501184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae87dd-516b-4818-8fd1-c5f99f000a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:09.502845 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae87dd-516b-4818-8fd1-c5f99f000a90 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:09.512900 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1218/2411] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:00:09.517746 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b0051-d7e0-40d7-a19e-869f16cf2139 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:09.527611 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b0051-d7e0-40d7-a19e-869f16cf2139 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:09.532273 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d7e9ce-f681-435e-9092-52fffe87a84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:09.532896 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7e9ce-f681-435e-9092-52fffe87a84f admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:09.533180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7e9ce-f681-435e-9092-52fffe87a84f admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:09.533455 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7e9ce-f681-435e-9092-52fffe87a84f admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:09.534087 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d7e9ce-f681-435e-9092-52fffe87a84f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:09.535686 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d7e9ce-f681-435e-9092-52fffe87a84f admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:09.546125 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1194/2412] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:00:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:00:09.550921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae87dd-516b-4818-8fd1-c5f99f000a90 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:09.560770 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae87dd-516b-4818-8fd1-c5f99f000a90 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:09.565930 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77190e-8a3a-4975-91b3-252be1547aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:09.566578 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77190e-8a3a-4975-91b3-252be1547aa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:09.566858 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77190e-8a3a-4975-91b3-252be1547aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:09.567132 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77190e-8a3a-4975-91b3-252be1547aa1 admin admin] PATH_INFO: `/v3/projects/fede7ffbdd1d4ed8bbc37b55e03e4791/users/b8a999d19a634bcfb26d2d936b3429bc/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:09.581352 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77190e-8a3a-4975-91b3-252be1547aa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fede7ffbdd1d4ed8bbc37b55e03e4791, user_id=b8a999d19a634bcfb26d2d936b3429bc, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:09.583979 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77190e-8a3a-4975-91b3-252be1547aa1 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:09.597023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1219/2413] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:00:09 2022] PUT /identity/v3/projects/fede7ffbdd1d4ed8bbc37b55e03e4791/users/b8a999d19a634bcfb26d2d936b3429bc/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:00:09.601935 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2f277-e74b-4f35-8611-285eba32e0f7 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:09.602219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2f277-e74b-4f35-8611-285eba32e0f7 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:09.602492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2f277-e74b-4f35-8611-285eba32e0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:09.622031 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-eab2f277-e74b-4f35-8611-285eba32e0f7 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:00:09.623247 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-eab2f277-e74b-4f35-8611-285eba32e0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:09.682596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1195/2414] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:00:09.693556 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77190e-8a3a-4975-91b3-252be1547aa1 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:09.695261 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-7c77190e-8a3a-4975-91b3-252be1547aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:09.717512 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:09.756289 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77190e-8a3a-4975-91b3-252be1547aa1 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:09.759816 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9bb45-8407-45da-a5f1-d61f65e0c9b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:09.760448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9bb45-8407-45da-a5f1-d61f65e0c9b0 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:09.760726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9bb45-8407-45da-a5f1-d61f65e0c9b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:09.760995 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9bb45-8407-45da-a5f1-d61f65e0c9b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:09.770348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9bb45-8407-45da-a5f1-d61f65e0c9b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:09.771761 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9bb45-8407-45da-a5f1-d61f65e0c9b0 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:09.781097 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1220/2415] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:00:13.672925 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88155670-5afc-4f27-8490-154cbe69b620 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:13.673448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88155670-5afc-4f27-8490-154cbe69b620 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:13.673995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88155670-5afc-4f27-8490-154cbe69b620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:13.706663 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-88155670-5afc-4f27-8490-154cbe69b620 None None] MFA Rules not processed for user `b8a999d19a634bcfb26d2d936b3429bc`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:00:13.707952 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-88155670-5afc-4f27-8490-154cbe69b620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:13.726258 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:13.780484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1196/2416] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:00:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:00:13.785176 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9bb45-8407-45da-a5f1-d61f65e0c9b0 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:13.801301 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9bb45-8407-45da-a5f1-d61f65e0c9b0 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:13.817438 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bc069-3092-4609-b1aa-b4d453de42c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:13.819769 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bc069-3092-4609-b1aa-b4d453de42c8 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:13.822349 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bc069-3092-4609-b1aa-b4d453de42c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:13.822756 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bc069-3092-4609-b1aa-b4d453de42c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:13.823539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bc069-3092-4609-b1aa-b4d453de42c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:13.827884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bc069-3092-4609-b1aa-b4d453de42c8 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:13.862296 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1221/2417] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 01:00:13 2022] POST /identity/v3/projects => generated 371 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:00:13.866271 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88155670-5afc-4f27-8490-154cbe69b620 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:13.882395 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88155670-5afc-4f27-8490-154cbe69b620 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:13.948092 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9adf8-81f2-43a8-a3b4-b2fed428d77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:13.948842 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9adf8-81f2-43a8-a3b4-b2fed428d77b admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:13.949123 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9adf8-81f2-43a8-a3b4-b2fed428d77b admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:13.949392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9adf8-81f2-43a8-a3b4-b2fed428d77b admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:13.950061 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9adf8-81f2-43a8-a3b4-b2fed428d77b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:13.951992 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9adf8-81f2-43a8-a3b4-b2fed428d77b admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:13.974969 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1197/2418] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 01:00:13 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:00:13.978989 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bc069-3092-4609-b1aa-b4d453de42c8 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:13.991084 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bc069-3092-4609-b1aa-b4d453de42c8 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:13.995940 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81646bb8-083a-451e-9d2e-e9c8390c9cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:13.996584 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81646bb8-083a-451e-9d2e-e9c8390c9cfa admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:13.996849 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81646bb8-083a-451e-9d2e-e9c8390c9cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:13.997125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81646bb8-083a-451e-9d2e-e9c8390c9cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:13.997764 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81646bb8-083a-451e-9d2e-e9c8390c9cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:13.999374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81646bb8-083a-451e-9d2e-e9c8390c9cfa admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:14.012162 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1222/2419] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:00:14.017158 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9adf8-81f2-43a8-a3b4-b2fed428d77b admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:14.027678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9adf8-81f2-43a8-a3b4-b2fed428d77b admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:14.033928 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a467af9-e0f3-45c5-b8d1-b30cf94bb532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:14.034727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a467af9-e0f3-45c5-b8d1-b30cf94bb532 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:14.034872 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a467af9-e0f3-45c5-b8d1-b30cf94bb532 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:14.035153 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a467af9-e0f3-45c5-b8d1-b30cf94bb532 admin admin] PATH_INFO: `/v3/projects/57f6ac6d20bf40ba8fcfbdd9a21c4d40/users/b8b38cddd5a248d8bab0d7a6482d3b2f/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:14.053256 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a467af9-e0f3-45c5-b8d1-b30cf94bb532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f6ac6d20bf40ba8fcfbdd9a21c4d40, user_id=b8b38cddd5a248d8bab0d7a6482d3b2f, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:14.055880 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a467af9-e0f3-45c5-b8d1-b30cf94bb532 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:14.069912 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1198/2420] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:00:14 2022] PUT /identity/v3/projects/57f6ac6d20bf40ba8fcfbdd9a21c4d40/users/b8b38cddd5a248d8bab0d7a6482d3b2f/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:00:14.073888 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81646bb8-083a-451e-9d2e-e9c8390c9cfa admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:14.084632 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81646bb8-083a-451e-9d2e-e9c8390c9cfa admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:14.144455 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739034c5-f678-4260-bc34-738af347d93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:14.145245 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739034c5-f678-4260-bc34-738af347d93a admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:14.145348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739034c5-f678-4260-bc34-738af347d93a admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:14.145613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739034c5-f678-4260-bc34-738af347d93a admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:14.146255 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739034c5-f678-4260-bc34-738af347d93a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:14.147863 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739034c5-f678-4260-bc34-738af347d93a admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:14.159516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1223/2421] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:00:14.163647 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a467af9-e0f3-45c5-b8d1-b30cf94bb532 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:14.174779 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a467af9-e0f3-45c5-b8d1-b30cf94bb532 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:14.179752 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402e3e0-2d04-4e86-a854-c64c01cd6329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:14.180491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402e3e0-2d04-4e86-a854-c64c01cd6329 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:14.180774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402e3e0-2d04-4e86-a854-c64c01cd6329 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:14.181056 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402e3e0-2d04-4e86-a854-c64c01cd6329 admin admin] PATH_INFO: `/v3/projects/57f6ac6d20bf40ba8fcfbdd9a21c4d40/users/b8b38cddd5a248d8bab0d7a6482d3b2f/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:14.183365 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402e3e0-2d04-4e86-a854-c64c01cd6329 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f6ac6d20bf40ba8fcfbdd9a21c4d40, user_id=b8b38cddd5a248d8bab0d7a6482d3b2f, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:14.186129 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402e3e0-2d04-4e86-a854-c64c01cd6329 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:14.197537 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1199/2422] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:00:14 2022] PUT /identity/v3/projects/57f6ac6d20bf40ba8fcfbdd9a21c4d40/users/b8b38cddd5a248d8bab0d7a6482d3b2f/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:00:18.575818 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7dba-67ad-4104-9699-634696446fa3 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:18.576371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7dba-67ad-4104-9699-634696446fa3 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:18.576371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7dba-67ad-4104-9699-634696446fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:18.606052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-fe3f7dba-67ad-4104-9699-634696446fa3 None None] MFA Rules not processed for user `b8b38cddd5a248d8bab0d7a6482d3b2f`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:00:18.607527 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-fe3f7dba-67ad-4104-9699-634696446fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:18.680138 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1224/2423] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:00:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:00:18.699366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402e3e0-2d04-4e86-a854-c64c01cd6329 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:18.701441 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-0402e3e0-2d04-4e86-a854-c64c01cd6329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:18.788751 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402e3e0-2d04-4e86-a854-c64c01cd6329 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:18.794663 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44728ee2-f52b-44ac-be31-da262dba167d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:18.795284 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44728ee2-f52b-44ac-be31-da262dba167d service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:18.795584 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44728ee2-f52b-44ac-be31-da262dba167d service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:18.795851 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44728ee2-f52b-44ac-be31-da262dba167d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:18.806314 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:18.862471 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44728ee2-f52b-44ac-be31-da262dba167d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:18.864101 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44728ee2-f52b-44ac-be31-da262dba167d service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:18.873516 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1200/2424] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:00:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:00:19.940371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a2e84-7ae2-48f4-9278-97110629b7df None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:19.940852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a2e84-7ae2-48f4-9278-97110629b7df None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:19.940938 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a2e84-7ae2-48f4-9278-97110629b7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:19.962028 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-6a7a2e84-7ae2-48f4-9278-97110629b7df None None] MFA Rules not processed for user `b8a999d19a634bcfb26d2d936b3429bc`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:00:19.963314 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-6a7a2e84-7ae2-48f4-9278-97110629b7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:19.970593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1225/2425] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:00:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:00:28.869185 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=24 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:29.956177 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:40.810185 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44728ee2-f52b-44ac-be31-da262dba167d service nova] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:40.865578 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44728ee2-f52b-44ac-be31-da262dba167d service nova] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:40.869201 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a375da4-b71c-4337-b59c-7779546864c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:40.869801 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a375da4-b71c-4337-b59c-7779546864c5 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:40.870079 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a375da4-b71c-4337-b59c-7779546864c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:40.870349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a375da4-b71c-4337-b59c-7779546864c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:40.872071 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-9a375da4-b71c-4337-b59c-7779546864c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:40.882828 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a375da4-b71c-4337-b59c-7779546864c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:40.884294 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a375da4-b71c-4337-b59c-7779546864c5 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:40.892244 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1201/2426] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:00:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:00:43.831199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a2e84-7ae2-48f4-9278-97110629b7df None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:43.832708 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-6a7a2e84-7ae2-48f4-9278-97110629b7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:43.851704 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a2e84-7ae2-48f4-9278-97110629b7df None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:43.857378 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97f045-6160-415b-8f7a-3037f3caca39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:43.857978 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97f045-6160-415b-8f7a-3037f3caca39 service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:43.858262 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97f045-6160-415b-8f7a-3037f3caca39 service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:43.858533 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97f045-6160-415b-8f7a-3037f3caca39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:43.860241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-7c97f045-6160-415b-8f7a-3037f3caca39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:43.886451 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97f045-6160-415b-8f7a-3037f3caca39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:43.888046 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97f045-6160-415b-8f7a-3037f3caca39 service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:43.900868 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1226/2427] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:00:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:00:50.888608 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=26 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:53.895914 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=6 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:00:55.236350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a375da4-b71c-4337-b59c-7779546864c5 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:55.251310 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a375da4-b71c-4337-b59c-7779546864c5 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:55.257662 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de348e-f7fd-4a84-ae75-20b85491ed49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:55.258293 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de348e-f7fd-4a84-ae75-20b85491ed49 service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:55.258568 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de348e-f7fd-4a84-ae75-20b85491ed49 service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:55.258839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de348e-f7fd-4a84-ae75-20b85491ed49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:55.260598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b4de348e-f7fd-4a84-ae75-20b85491ed49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:55.274843 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de348e-f7fd-4a84-ae75-20b85491ed49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:55.276554 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de348e-f7fd-4a84-ae75-20b85491ed49 service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:55.287996 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1202/2428] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:00:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:00:58.205855 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97f045-6160-415b-8f7a-3037f3caca39 service nova] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:58.215998 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97f045-6160-415b-8f7a-3037f3caca39 service nova] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:58.219615 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25322193-7192-4bc0-9d5c-c296597a392b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:58.220250 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25322193-7192-4bc0-9d5c-c296597a392b service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:58.220531 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25322193-7192-4bc0-9d5c-c296597a392b service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:58.220802 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25322193-7192-4bc0-9d5c-c296597a392b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:58.231144 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25322193-7192-4bc0-9d5c-c296597a392b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:58.232639 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25322193-7192-4bc0-9d5c-c296597a392b service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:58.242912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1227/2429] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:00:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:00:59.843524 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de348e-f7fd-4a84-ae75-20b85491ed49 service nova] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:00:59.855959 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de348e-f7fd-4a84-ae75-20b85491ed49 service nova] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:00:59.861966 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19211571-0614-4fc4-8835-61eafee581e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:00:59.862579 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19211571-0614-4fc4-8835-61eafee581e5 service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:00:59.863169 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19211571-0614-4fc4-8835-61eafee581e5 service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:00:59.863480 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19211571-0614-4fc4-8835-61eafee581e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:00:59.865273 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-19211571-0614-4fc4-8835-61eafee581e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:00:59.880208 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19211571-0614-4fc4-8835-61eafee581e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:00:59.883054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19211571-0614-4fc4-8835-61eafee581e5 service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:00:59.914180 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1203/2430] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:00:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:01:01.359923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98bbce-7fe6-477c-90fe-37c5acbb7acf None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:01:01.360539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98bbce-7fe6-477c-90fe-37c5acbb7acf None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:01:01.360539 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98bbce-7fe6-477c-90fe-37c5acbb7acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:01:01.380253 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-5b98bbce-7fe6-477c-90fe-37c5acbb7acf None None] MFA Rules not processed for user `7353722ccb7548a6b92c89d4917fefaa`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:01:01.381860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-5b98bbce-7fe6-477c-90fe-37c5acbb7acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:01:01.437776 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1228/2431] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:01:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:01:09.896234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=12 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:01:11.430940 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=31 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:19.370365 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19211571-0614-4fc4-8835-61eafee581e5 service nova] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:02:19.371815 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-19211571-0614-4fc4-8835-61eafee581e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:19.441582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19211571-0614-4fc4-8835-61eafee581e5 service nova] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:02:19.446462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d05e36-5671-4c07-912f-52d6c8dda1d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:02:19.447059 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d05e36-5671-4c07-912f-52d6c8dda1d5 service placement] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:19.447447 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d05e36-5671-4c07-912f-52d6c8dda1d5 service placement] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:19.447725 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d05e36-5671-4c07-912f-52d6c8dda1d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:19.449461 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d9d05e36-5671-4c07-912f-52d6c8dda1d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:19.461151 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d05e36-5671-4c07-912f-52d6c8dda1d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:02:19.462726 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d05e36-5671-4c07-912f-52d6c8dda1d5 service placement] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:02:19.473610 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1204/2432] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:02:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:02:29.468291 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=32 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:02:40.318234 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f12aa-1ddb-473f-8418-20651f1aa274 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:02:40.318764 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f12aa-1ddb-473f-8418-20651f1aa274 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:02:40.318764 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f12aa-1ddb-473f-8418-20651f1aa274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:02:40.338220 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-c48f12aa-1ddb-473f-8418-20651f1aa274 None None] MFA Rules not processed for user `7353722ccb7548a6b92c89d4917fefaa`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:02:40.339426 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c48f12aa-1ddb-473f-8418-20651f1aa274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:02:40.346922 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1229/2433] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:02:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:02:50.332847 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:26.556294 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d05e36-5671-4c07-912f-52d6c8dda1d5 service placement] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:03:26.565722 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d05e36-5671-4c07-912f-52d6c8dda1d5 service placement] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:03:26.569177 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585e387-900c-40b5-ba36-9d432c4e26fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:03:26.569842 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585e387-900c-40b5-ba36-9d432c4e26fc service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:26.570125 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585e387-900c-40b5-ba36-9d432c4e26fc service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:26.570392 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585e387-900c-40b5-ba36-9d432c4e26fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:26.572107 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-6585e387-900c-40b5-ba36-9d432c4e26fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:26.585761 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6585e387-900c-40b5-ba36-9d432c4e26fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:03:26.587173 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6585e387-900c-40b5-ba36-9d432c4e26fc service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:03:26.598069 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1205/2434] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:03:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:03:26.908799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f12aa-1ddb-473f-8418-20651f1aa274 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:03:26.910378 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-c48f12aa-1ddb-473f-8418-20651f1aa274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:26.923710 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f12aa-1ddb-473f-8418-20651f1aa274 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:03:26.927382 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce41ea-038d-4e7d-a207-65ab743aac8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:03:26.927988 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce41ea-038d-4e7d-a207-65ab743aac8b service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:26.928340 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce41ea-038d-4e7d-a207-65ab743aac8b service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:26.930490 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce41ea-038d-4e7d-a207-65ab743aac8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:26.932188 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-82ce41ea-038d-4e7d-a207-65ab743aac8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:26.942261 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce41ea-038d-4e7d-a207-65ab743aac8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:03:26.944184 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce41ea-038d-4e7d-a207-65ab743aac8b service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:03:26.953477 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1230/2435] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:03:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:03:27.043226 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585e387-900c-40b5-ba36-9d432c4e26fc service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:03:27.051936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585e387-900c-40b5-ba36-9d432c4e26fc service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:03:27.056623 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eda6bd-b43b-448f-8528-6f24b3cb4652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:03:27.057248 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eda6bd-b43b-448f-8528-6f24b3cb4652 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:27.057507 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eda6bd-b43b-448f-8528-6f24b3cb4652 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:27.057784 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eda6bd-b43b-448f-8528-6f24b3cb4652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:27.059488 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b7eda6bd-b43b-448f-8528-6f24b3cb4652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:27.072905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eda6bd-b43b-448f-8528-6f24b3cb4652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:03:27.074336 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eda6bd-b43b-448f-8528-6f24b3cb4652 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:03:27.085385 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1206/2436] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:03:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:03:36.372211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce41ea-038d-4e7d-a207-65ab743aac8b service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:03:36.381986 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce41ea-038d-4e7d-a207-65ab743aac8b service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:03:36.385625 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd0195-903f-4ceb-b96c-db7abe46a922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:03:36.386228 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd0195-903f-4ceb-b96c-db7abe46a922 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:03:36.386503 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd0195-903f-4ceb-b96c-db7abe46a922 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:03:36.386773 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd0195-903f-4ceb-b96c-db7abe46a922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:03:36.388530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-73dd0195-903f-4ceb-b96c-db7abe46a922 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:03:36.404876 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd0195-903f-4ceb-b96c-db7abe46a922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:03:36.406631 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd0195-903f-4ceb-b96c-db7abe46a922 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:03:36.419637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1231/2437] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:03:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:03:37.080386 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=12 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:03:46.412687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=12 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:05.059753 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eda6bd-b43b-448f-8528-6f24b3cb4652 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:04:05.071256 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eda6bd-b43b-448f-8528-6f24b3cb4652 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:04:05.076757 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40177f2-f19d-4b61-bb46-268b2e980fb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:04:05.077351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40177f2-f19d-4b61-bb46-268b2e980fb2 service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:04:05.077649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40177f2-f19d-4b61-bb46-268b2e980fb2 service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:04:05.077904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40177f2-f19d-4b61-bb46-268b2e980fb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:04:05.079610 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c40177f2-f19d-4b61-bb46-268b2e980fb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:04:05.090590 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40177f2-f19d-4b61-bb46-268b2e980fb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:04:05.092200 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40177f2-f19d-4b61-bb46-268b2e980fb2 service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:04:05.101577 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1207/2438] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:04:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:04:15.097473 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=6 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:21.523435 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2cc49-e998-4985-add8-13d12ef8b9c3 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:04:21.523959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2cc49-e998-4985-add8-13d12ef8b9c3 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:04:21.523959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2cc49-e998-4985-add8-13d12ef8b9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:04:21.544356 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-70a2cc49-e998-4985-add8-13d12ef8b9c3 None None] MFA Rules not processed for user `7353722ccb7548a6b92c89d4917fefaa`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:04:21.545564 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-70a2cc49-e998-4985-add8-13d12ef8b9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:04:21.552567 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1232/2439] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:04:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:04:21.859017 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40177f2-f19d-4b61-bb46-268b2e980fb2 service nova] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:04:21.868717 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40177f2-f19d-4b61-bb46-268b2e980fb2 service nova] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:04:21.872378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf6e3b-851d-4a6d-b2ca-44a6f86fb8e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:04:21.872968 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf6e3b-851d-4a6d-b2ca-44a6f86fb8e1 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:04:21.873244 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf6e3b-851d-4a6d-b2ca-44a6f86fb8e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:04:21.873511 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf6e3b-851d-4a6d-b2ca-44a6f86fb8e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:04:21.875226 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-cbaf6e3b-851d-4a6d-b2ca-44a6f86fb8e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:04:21.885785 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf6e3b-851d-4a6d-b2ca-44a6f86fb8e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:04:21.887233 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf6e3b-851d-4a6d-b2ca-44a6f86fb8e1 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:04:21.895997 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1208/2440] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:04:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:04:22.207023 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2cc49-e998-4985-add8-13d12ef8b9c3 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:04:22.219293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2cc49-e998-4985-add8-13d12ef8b9c3 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:04:22.225577 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f283f-4cc4-4b21-8bf0-64326363f9fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:04:22.226330 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f283f-4cc4-4b21-8bf0-64326363f9fa service placement] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:04:22.226613 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f283f-4cc4-4b21-8bf0-64326363f9fa service placement] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:04:22.226886 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f283f-4cc4-4b21-8bf0-64326363f9fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:04:22.229141 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-778f283f-4cc4-4b21-8bf0-64326363f9fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:04:22.242288 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f283f-4cc4-4b21-8bf0-64326363f9fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:04:22.243811 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f283f-4cc4-4b21-8bf0-64326363f9fa service placement] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:04:22.255243 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1233/2441] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:04:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:04:24.212748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf6e3b-851d-4a6d-b2ca-44a6f86fb8e1 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:04:24.222448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf6e3b-851d-4a6d-b2ca-44a6f86fb8e1 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:04:24.226584 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffc402-c5aa-4268-9838-6695eafed82c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:04:24.227291 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffc402-c5aa-4268-9838-6695eafed82c service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:04:24.227578 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffc402-c5aa-4268-9838-6695eafed82c service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:04:24.227856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffc402-c5aa-4268-9838-6695eafed82c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:04:24.229721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-36ffc402-c5aa-4268-9838-6695eafed82c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:04:24.241716 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ffc402-c5aa-4268-9838-6695eafed82c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:04:24.243183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ffc402-c5aa-4268-9838-6695eafed82c service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:04:24.252273 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1209/2442] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:04:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:04:32.249618 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=9 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:04:34.247969 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=12 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:12.570491 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f283f-4cc4-4b21-8bf0-64326363f9fa service placement] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:12.580785 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f283f-4cc4-4b21-8bf0-64326363f9fa service placement] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:12.584406 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a4048-af7b-4891-bc1e-2723f5349465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:12.584943 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a4048-af7b-4891-bc1e-2723f5349465 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:12.585227 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a4048-af7b-4891-bc1e-2723f5349465 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:12.585493 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a4048-af7b-4891-bc1e-2723f5349465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:12.646850 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a4048-af7b-4891-bc1e-2723f5349465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:12.648448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a4048-af7b-4891-bc1e-2723f5349465 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:12.658707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1234/2443] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:05:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:05:16.003005 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffc402-c5aa-4268-9838-6695eafed82c service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:16.005351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-36ffc402-c5aa-4268-9838-6695eafed82c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:16.024697 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffc402-c5aa-4268-9838-6695eafed82c service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:16.032713 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33167613-0f71-40dc-961e-0187790f8863 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:16.034137 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33167613-0f71-40dc-961e-0187790f8863 service placement] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:16.034906 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33167613-0f71-40dc-961e-0187790f8863 service placement] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:16.035289 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33167613-0f71-40dc-961e-0187790f8863 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:16.038408 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-33167613-0f71-40dc-961e-0187790f8863 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:16.067179 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33167613-0f71-40dc-961e-0187790f8863 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:16.068995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33167613-0f71-40dc-961e-0187790f8863 service placement] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:16.090433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1210/2444] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:05:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:05:19.524914 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a4048-af7b-4891-bc1e-2723f5349465 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:19.532407 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a4048-af7b-4891-bc1e-2723f5349465 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:19.537054 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a419ec-36fd-4111-beb5-87b89a21562f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:19.537689 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a419ec-36fd-4111-beb5-87b89a21562f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:19.538578 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a419ec-36fd-4111-beb5-87b89a21562f admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:19.538578 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a419ec-36fd-4111-beb5-87b89a21562f admin admin] PATH_INFO: `/v3/users/b8a999d19a634bcfb26d2d936b3429bc` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:19.539335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a419ec-36fd-4111-beb5-87b89a21562f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8a999d19a634bcfb26d2d936b3429bc)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:19.541307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a419ec-36fd-4111-beb5-87b89a21562f admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:19.590441 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-68a419ec-36fd-4111-beb5-87b89a21562f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8a999d19a634bcfb26d2d936b3429bc'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:19.591099 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-68a419ec-36fd-4111-beb5-87b89a21562f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8a999d19a634bcfb26d2d936b3429bc'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:19.603330 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-68a419ec-36fd-4111-beb5-87b89a21562f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8a999d19a634bcfb26d2d936b3429bc'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:19.610407 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-68a419ec-36fd-4111-beb5-87b89a21562f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8a999d19a634bcfb26d2d936b3429bc'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:19.618113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-68a419ec-36fd-4111-beb5-87b89a21562f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8a999d19a634bcfb26d2d936b3429bc'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:19.623918 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1235/2445] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 01:05:19 2022] DELETE /identity/v3/users/b8a999d19a634bcfb26d2d936b3429bc => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:05:19.803628 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33167613-0f71-40dc-961e-0187790f8863 service placement] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:19.816547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33167613-0f71-40dc-961e-0187790f8863 service placement] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:19.873847 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c67b8a-3fd1-469f-b0a5-c515a09db79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:19.874486 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c67b8a-3fd1-469f-b0a5-c515a09db79c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:19.876274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c67b8a-3fd1-469f-b0a5-c515a09db79c admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:19.876274 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c67b8a-3fd1-469f-b0a5-c515a09db79c admin admin] PATH_INFO: `/v3/projects/fede7ffbdd1d4ed8bbc37b55e03e4791` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:19.876716 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c67b8a-3fd1-469f-b0a5-c515a09db79c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fede7ffbdd1d4ed8bbc37b55e03e4791)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:19.878633 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c67b8a-3fd1-469f-b0a5-c515a09db79c admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:19.928186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-37c67b8a-3fd1-469f-b0a5-c515a09db79c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fede7ffbdd1d4ed8bbc37b55e03e4791', 'request_id': 'req-37c67b8a-3fd1-469f-b0a5-c515a09db79c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:19.935522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-37c67b8a-3fd1-469f-b0a5-c515a09db79c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fede7ffbdd1d4ed8bbc37b55e03e4791', 'request_id': 'req-37c67b8a-3fd1-469f-b0a5-c515a09db79c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:19.942719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-37c67b8a-3fd1-469f-b0a5-c515a09db79c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fede7ffbdd1d4ed8bbc37b55e03e4791', 'request_id': 'req-37c67b8a-3fd1-469f-b0a5-c515a09db79c'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:19.953285 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-37c67b8a-3fd1-469f-b0a5-c515a09db79c admin admin] The token cache is being invalidate because project fede7ffbdd1d4ed8bbc37b55e03e4791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:05:19.953575 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-37c67b8a-3fd1-469f-b0a5-c515a09db79c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:19.954208 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-37c67b8a-3fd1-469f-b0a5-c515a09db79c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:19.955715 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1211/2446] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 01:05:19 2022] DELETE /identity/v3/projects/fede7ffbdd1d4ed8bbc37b55e03e4791 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:05:19.960212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a419ec-36fd-4111-beb5-87b89a21562f admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:19.961758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-68a419ec-36fd-4111-beb5-87b89a21562f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:19.974823 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a419ec-36fd-4111-beb5-87b89a21562f admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:20.030874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb25795-fdf2-4826-a283-ba3a456d622c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:20.031518 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb25795-fdf2-4826-a283-ba3a456d622c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:20.031798 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb25795-fdf2-4826-a283-ba3a456d622c admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:20.032104 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb25795-fdf2-4826-a283-ba3a456d622c admin admin] PATH_INFO: `/v3/users/b8b38cddd5a248d8bab0d7a6482d3b2f` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:20.033251 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb25795-fdf2-4826-a283-ba3a456d622c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8b38cddd5a248d8bab0d7a6482d3b2f)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:20.035191 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb25795-fdf2-4826-a283-ba3a456d622c admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:20.040822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:20.084171 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-dbb25795-fdf2-4826-a283-ba3a456d622c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8b38cddd5a248d8bab0d7a6482d3b2f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:20.084828 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-dbb25795-fdf2-4826-a283-ba3a456d622c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8b38cddd5a248d8bab0d7a6482d3b2f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:20.097665 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-dbb25795-fdf2-4826-a283-ba3a456d622c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8b38cddd5a248d8bab0d7a6482d3b2f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:20.104589 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-dbb25795-fdf2-4826-a283-ba3a456d622c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8b38cddd5a248d8bab0d7a6482d3b2f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:20.112893 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-dbb25795-fdf2-4826-a283-ba3a456d622c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8b38cddd5a248d8bab0d7a6482d3b2f'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:20.118771 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1236/2447] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 01:05:19 2022] DELETE /identity/v3/users/b8b38cddd5a248d8bab0d7a6482d3b2f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:05:20.289175 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c67b8a-3fd1-469f-b0a5-c515a09db79c admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:20.290614 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-37c67b8a-3fd1-469f-b0a5-c515a09db79c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:20.304106 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c67b8a-3fd1-469f-b0a5-c515a09db79c admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:20.365432 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d2d28-764e-466a-a0b7-3a11d6c02c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:20.366069 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d2d28-764e-466a-a0b7-3a11d6c02c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:20.366351 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d2d28-764e-466a-a0b7-3a11d6c02c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:20.366624 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d2d28-764e-466a-a0b7-3a11d6c02c5e admin admin] PATH_INFO: `/v3/projects/57f6ac6d20bf40ba8fcfbdd9a21c4d40` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:20.367743 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d2d28-764e-466a-a0b7-3a11d6c02c5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=57f6ac6d20bf40ba8fcfbdd9a21c4d40)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:20.369693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d2d28-764e-466a-a0b7-3a11d6c02c5e admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:20.390285 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:20.417377 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-fe5d2d28-764e-466a-a0b7-3a11d6c02c5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57f6ac6d20bf40ba8fcfbdd9a21c4d40', 'request_id': 'req-fe5d2d28-764e-466a-a0b7-3a11d6c02c5e'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:20.426103 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-fe5d2d28-764e-466a-a0b7-3a11d6c02c5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57f6ac6d20bf40ba8fcfbdd9a21c4d40', 'request_id': 'req-fe5d2d28-764e-466a-a0b7-3a11d6c02c5e'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:20.433165 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-fe5d2d28-764e-466a-a0b7-3a11d6c02c5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57f6ac6d20bf40ba8fcfbdd9a21c4d40', 'request_id': 'req-fe5d2d28-764e-466a-a0b7-3a11d6c02c5e'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:20.443946 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-fe5d2d28-764e-466a-a0b7-3a11d6c02c5e admin admin] The token cache is being invalidate because project 57f6ac6d20bf40ba8fcfbdd9a21c4d40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:05:20.444267 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-fe5d2d28-764e-466a-a0b7-3a11d6c02c5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:20.444906 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-fe5d2d28-764e-466a-a0b7-3a11d6c02c5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:05:20.446377 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1212/2448] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 01:05:20 2022] DELETE /identity/v3/projects/57f6ac6d20bf40ba8fcfbdd9a21c4d40 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:05:20.451404 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080d8c7-6265-419e-8218-9101a349d70b None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:20.451711 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080d8c7-6265-419e-8218-9101a349d70b None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:20.451984 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080d8c7-6265-419e-8218-9101a349d70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:20.473074 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-8080d8c7-6265-419e-8218-9101a349d70b None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:05:20.474266 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-8080d8c7-6265-419e-8218-9101a349d70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:20.534006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1237/2449] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:05:20.543061 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044a6e6-85b0-4f12-a799-d3dd4296ed43 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:20.543350 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044a6e6-85b0-4f12-a799-d3dd4296ed43 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:20.543631 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044a6e6-85b0-4f12-a799-d3dd4296ed43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:20.564800 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-b044a6e6-85b0-4f12-a799-d3dd4296ed43 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:05:20.566100 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b044a6e6-85b0-4f12-a799-d3dd4296ed43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:20.574138 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1213/2450] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:05:20.579192 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080d8c7-6265-419e-8218-9101a349d70b None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:20.588833 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080d8c7-6265-419e-8218-9101a349d70b None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:20.594053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d0968-44dc-44ac-8858-1e4eef2b6e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:20.594744 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d0968-44dc-44ac-8858-1e4eef2b6e54 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:20.595018 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d0968-44dc-44ac-8858-1e4eef2b6e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:20.595294 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d0968-44dc-44ac-8858-1e4eef2b6e54 admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:20.595932 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d0968-44dc-44ac-8858-1e4eef2b6e54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:05:20.596266 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d0968-44dc-44ac-8858-1e4eef2b6e54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:20.597852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d0968-44dc-44ac-8858-1e4eef2b6e54 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:20.609731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1238/2451] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 01:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:05:20.613915 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044a6e6-85b0-4f12-a799-d3dd4296ed43 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:20.622921 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044a6e6-85b0-4f12-a799-d3dd4296ed43 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:20.627552 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cbb7c1-4a1a-4364-89e9-55a5404ce5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:20.628173 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbb7c1-4a1a-4364-89e9-55a5404ce5b0 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:20.628449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbb7c1-4a1a-4364-89e9-55a5404ce5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:20.628719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbb7c1-4a1a-4364-89e9-55a5404ce5b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:20.629338 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cbb7c1-4a1a-4364-89e9-55a5404ce5b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:20.631217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cbb7c1-4a1a-4364-89e9-55a5404ce5b0 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:20.659536 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1214/2452] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 01:05:20 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:05:20.663627 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d0968-44dc-44ac-8858-1e4eef2b6e54 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:20.673866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d0968-44dc-44ac-8858-1e4eef2b6e54 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:20.729636 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fda61-67dd-4adc-aef4-84df59669570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:20.730241 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fda61-67dd-4adc-aef4-84df59669570 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:20.730519 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fda61-67dd-4adc-aef4-84df59669570 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:20.730793 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fda61-67dd-4adc-aef4-84df59669570 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:20.731427 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267fda61-67dd-4adc-aef4-84df59669570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:20.733368 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267fda61-67dd-4adc-aef4-84df59669570 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:20.754636 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1239/2453] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 01:05:20 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:05:20.758902 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cbb7c1-4a1a-4364-89e9-55a5404ce5b0 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:20.769879 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cbb7c1-4a1a-4364-89e9-55a5404ce5b0 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:20.774441 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63975655-1add-43af-ac5d-c02c5e290a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:20.775028 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63975655-1add-43af-ac5d-c02c5e290a5a admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:20.775302 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63975655-1add-43af-ac5d-c02c5e290a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:20.775571 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63975655-1add-43af-ac5d-c02c5e290a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:20.776235 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63975655-1add-43af-ac5d-c02c5e290a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:20.777822 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63975655-1add-43af-ac5d-c02c5e290a5a admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:20.789013 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1215/2454] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:05:20.792831 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fda61-67dd-4adc-aef4-84df59669570 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:20.798068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:20.804818 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fda61-67dd-4adc-aef4-84df59669570 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:20.810216 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f840a11-21ee-4890-92d1-541109cff4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:20.810811 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f840a11-21ee-4890-92d1-541109cff4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:20.811093 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f840a11-21ee-4890-92d1-541109cff4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:20.811361 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f840a11-21ee-4890-92d1-541109cff4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:20.811991 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f840a11-21ee-4890-92d1-541109cff4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:20.813641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f840a11-21ee-4890-92d1-541109cff4d9 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:20.824335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1240/2455] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:05:20.829367 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63975655-1add-43af-ac5d-c02c5e290a5a admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:20.839143 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63975655-1add-43af-ac5d-c02c5e290a5a admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:20.844358 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b24e16-c5ce-4b21-9e0d-88b5db7ac299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:20.845000 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b24e16-c5ce-4b21-9e0d-88b5db7ac299 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:20.845283 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b24e16-c5ce-4b21-9e0d-88b5db7ac299 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:20.845556 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b24e16-c5ce-4b21-9e0d-88b5db7ac299 admin admin] PATH_INFO: `/v3/projects/a8a24adf04bb4adfbc7347a5e4ec6b52/users/d6d3fe4f3fbb4dedb7a0703a2bb6884d/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:20.859825 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b24e16-c5ce-4b21-9e0d-88b5db7ac299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a24adf04bb4adfbc7347a5e4ec6b52, user_id=d6d3fe4f3fbb4dedb7a0703a2bb6884d, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:20.862492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b24e16-c5ce-4b21-9e0d-88b5db7ac299 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:20.880108 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1216/2456] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:05:20 2022] PUT /identity/v3/projects/a8a24adf04bb4adfbc7347a5e4ec6b52/users/d6d3fe4f3fbb4dedb7a0703a2bb6884d/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:05:20.885153 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4826da-5e56-4229-9cde-07d94438e3f2 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:20.885459 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4826da-5e56-4229-9cde-07d94438e3f2 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:20.885735 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4826da-5e56-4229-9cde-07d94438e3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:20.905227 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-8e4826da-5e56-4229-9cde-07d94438e3f2 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:05:20.906545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-8e4826da-5e56-4229-9cde-07d94438e3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:20.965455 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1241/2457] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:05:20.975087 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b24e16-c5ce-4b21-9e0d-88b5db7ac299 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:20.976847 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-73b24e16-c5ce-4b21-9e0d-88b5db7ac299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:21.033505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b24e16-c5ce-4b21-9e0d-88b5db7ac299 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:21.037120 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469c303-a91a-4115-94ed-a1df0d91a376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:21.037725 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469c303-a91a-4115-94ed-a1df0d91a376 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:21.038005 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469c303-a91a-4115-94ed-a1df0d91a376 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:21.038284 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469c303-a91a-4115-94ed-a1df0d91a376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:21.047187 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469c303-a91a-4115-94ed-a1df0d91a376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:21.048599 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469c303-a91a-4115-94ed-a1df0d91a376 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:21.057936 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1217/2458] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:05:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:05:24.774414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c29259-da79-41b9-b261-a35994f6688e None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:24.774913 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c29259-da79-41b9-b261-a35994f6688e None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:24.775036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c29259-da79-41b9-b261-a35994f6688e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:24.805479 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-55c29259-da79-41b9-b261-a35994f6688e None None] MFA Rules not processed for user `d6d3fe4f3fbb4dedb7a0703a2bb6884d`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:05:24.807892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-55c29259-da79-41b9-b261-a35994f6688e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:24.877340 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1242/2459] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:05:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:05:24.881949 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469c303-a91a-4115-94ed-a1df0d91a376 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:24.896952 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469c303-a91a-4115-94ed-a1df0d91a376 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:24.901793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f683df-c40b-4e55-b675-b89122bcc1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:24.902486 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f683df-c40b-4e55-b675-b89122bcc1f8 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:24.902834 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f683df-c40b-4e55-b675-b89122bcc1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:24.903179 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f683df-c40b-4e55-b675-b89122bcc1f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:24.903887 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f683df-c40b-4e55-b675-b89122bcc1f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:24.906440 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f683df-c40b-4e55-b675-b89122bcc1f8 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:24.935609 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1218/2460] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 01:05:24 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:05:24.940365 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c29259-da79-41b9-b261-a35994f6688e None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:24.951631 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c29259-da79-41b9-b261-a35994f6688e None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:25.019465 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda441d-549b-42f0-afd4-c746d80dc78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:25.020118 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda441d-549b-42f0-afd4-c746d80dc78b admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:25.020397 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda441d-549b-42f0-afd4-c746d80dc78b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:25.020671 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda441d-549b-42f0-afd4-c746d80dc78b admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:25.021307 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda441d-549b-42f0-afd4-c746d80dc78b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:25.023251 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda441d-549b-42f0-afd4-c746d80dc78b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:25.047408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1243/2461] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 01:05:24 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:05:25.051388 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f683df-c40b-4e55-b675-b89122bcc1f8 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:25.062828 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f683df-c40b-4e55-b675-b89122bcc1f8 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:25.067458 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a93c74-7806-4311-ad22-8a526efb1ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:25.068300 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a93c74-7806-4311-ad22-8a526efb1ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:25.068582 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a93c74-7806-4311-ad22-8a526efb1ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:25.068854 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a93c74-7806-4311-ad22-8a526efb1ec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:25.069661 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a93c74-7806-4311-ad22-8a526efb1ec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:25.071428 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a93c74-7806-4311-ad22-8a526efb1ec1 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:25.082547 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1219/2462] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:05:25.088368 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda441d-549b-42f0-afd4-c746d80dc78b admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:25.100213 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda441d-549b-42f0-afd4-c746d80dc78b admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:25.104939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b3ab3-e286-42ea-9034-3e00b4c536b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:25.105753 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b3ab3-e286-42ea-9034-3e00b4c536b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:25.106125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b3ab3-e286-42ea-9034-3e00b4c536b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:25.106408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b3ab3-e286-42ea-9034-3e00b4c536b3 admin admin] PATH_INFO: `/v3/projects/0c9825d2e86e4f268a95aad241d43c3e/users/9d35d370e5fe4fe9955c348319c24103/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:25.116341 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:25.124735 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b3ab3-e286-42ea-9034-3e00b4c536b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c9825d2e86e4f268a95aad241d43c3e, user_id=9d35d370e5fe4fe9955c348319c24103, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:25.127652 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b3ab3-e286-42ea-9034-3e00b4c536b3 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:25.140948 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1244/2463] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:05:25 2022] PUT /identity/v3/projects/0c9825d2e86e4f268a95aad241d43c3e/users/9d35d370e5fe4fe9955c348319c24103/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:05:25.145012 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a93c74-7806-4311-ad22-8a526efb1ec1 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:25.154989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a93c74-7806-4311-ad22-8a526efb1ec1 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:25.164119 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:25.218836 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c08afa-47f2-452f-8426-364a6e8a8622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:25.219612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c08afa-47f2-452f-8426-364a6e8a8622 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:25.219897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c08afa-47f2-452f-8426-364a6e8a8622 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:25.220265 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c08afa-47f2-452f-8426-364a6e8a8622 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:25.220915 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c08afa-47f2-452f-8426-364a6e8a8622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:25.222578 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c08afa-47f2-452f-8426-364a6e8a8622 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:25.234450 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1220/2464] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:05:25.238581 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b3ab3-e286-42ea-9034-3e00b4c536b3 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:25.248877 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b3ab3-e286-42ea-9034-3e00b4c536b3 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:25.254090 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42fe16-8526-4c16-8fa3-cb6c9f945b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:25.254747 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42fe16-8526-4c16-8fa3-cb6c9f945b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:25.255026 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42fe16-8526-4c16-8fa3-cb6c9f945b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:25.255300 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42fe16-8526-4c16-8fa3-cb6c9f945b6d admin admin] PATH_INFO: `/v3/projects/0c9825d2e86e4f268a95aad241d43c3e/users/9d35d370e5fe4fe9955c348319c24103/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:25.257303 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42fe16-8526-4c16-8fa3-cb6c9f945b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c9825d2e86e4f268a95aad241d43c3e, user_id=9d35d370e5fe4fe9955c348319c24103, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:25.259903 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42fe16-8526-4c16-8fa3-cb6c9f945b6d admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:25.271009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1245/2465] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:05:25 2022] PUT /identity/v3/projects/0c9825d2e86e4f268a95aad241d43c3e/users/9d35d370e5fe4fe9955c348319c24103/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:05:29.857707 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579d205-e916-45eb-a8a9-18c533f67445 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:29.858953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579d205-e916-45eb-a8a9-18c533f67445 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:29.858953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579d205-e916-45eb-a8a9-18c533f67445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:29.894329 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-3579d205-e916-45eb-a8a9-18c533f67445 None None] MFA Rules not processed for user `9d35d370e5fe4fe9955c348319c24103`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:05:29.895991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-3579d205-e916-45eb-a8a9-18c533f67445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:29.993356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1221/2466] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:05:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:05:30.020593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42fe16-8526-4c16-8fa3-cb6c9f945b6d admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:30.023028 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-dc42fe16-8526-4c16-8fa3-cb6c9f945b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:30.117399 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42fe16-8526-4c16-8fa3-cb6c9f945b6d admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:30.123367 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bddcd-605d-4a87-817f-59c7686e79b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:30.124693 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bddcd-605d-4a87-817f-59c7686e79b7 service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:30.124693 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bddcd-605d-4a87-817f-59c7686e79b7 service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:30.125384 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bddcd-605d-4a87-817f-59c7686e79b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:30.193664 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bddcd-605d-4a87-817f-59c7686e79b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:30.195317 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bddcd-605d-4a87-817f-59c7686e79b7 service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:30.206624 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1246/2467] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:05:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:05:39.988917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=57 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:40.200917 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=66,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:05:53.911735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a7c12-26b1-4d9a-92fc-244667220055 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:53.912279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a7c12-26b1-4d9a-92fc-244667220055 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:53.912388 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a7c12-26b1-4d9a-92fc-244667220055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:53.934758 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-174a7c12-26b1-4d9a-92fc-244667220055 None None] MFA Rules not processed for user `d6d3fe4f3fbb4dedb7a0703a2bb6884d`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:05:53.935943 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-174a7c12-26b1-4d9a-92fc-244667220055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:53.943342 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1222/2468] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:05:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:05:54.261317 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bddcd-605d-4a87-817f-59c7686e79b7 service nova] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:54.262724 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-098bddcd-605d-4a87-817f-59c7686e79b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:54.320012 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bddcd-605d-4a87-817f-59c7686e79b7 service nova] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:54.323562 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa8f25-cd7b-4202-af62-cd6c4e60b02c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:54.324191 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa8f25-cd7b-4202-af62-cd6c4e60b02c service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:54.324469 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa8f25-cd7b-4202-af62-cd6c4e60b02c service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:54.324990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa8f25-cd7b-4202-af62-cd6c4e60b02c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:54.326648 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-92fa8f25-cd7b-4202-af62-cd6c4e60b02c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:54.336914 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa8f25-cd7b-4202-af62-cd6c4e60b02c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:54.338328 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa8f25-cd7b-4202-af62-cd6c4e60b02c service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:54.347379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1247/2469] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:05:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:05:57.247144 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a7c12-26b1-4d9a-92fc-244667220055 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:05:57.248717 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-174a7c12-26b1-4d9a-92fc-244667220055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:57.266852 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a7c12-26b1-4d9a-92fc-244667220055 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:05:57.273447 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cbb88-7ecb-4a02-81a6-6d84859ca0ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:05:57.274057 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cbb88-7ecb-4a02-81a6-6d84859ca0ee service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:05:57.274468 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cbb88-7ecb-4a02-81a6-6d84859ca0ee service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:05:57.274751 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cbb88-7ecb-4a02-81a6-6d84859ca0ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:05:57.276535 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b40cbb88-7ecb-4a02-81a6-6d84859ca0ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:05:57.290195 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40cbb88-7ecb-4a02-81a6-6d84859ca0ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:05:57.291904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40cbb88-7ecb-4a02-81a6-6d84859ca0ee service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:05:57.305112 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1223/2470] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:05:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:06:04.342540 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=26 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:07.299645 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=9 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:10.803135 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa8f25-cd7b-4202-af62-cd6c4e60b02c service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:10.819290 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa8f25-cd7b-4202-af62-cd6c4e60b02c service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:10.825288 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931c054-3a7f-436e-8fe7-50047c71e4b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:10.825923 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931c054-3a7f-436e-8fe7-50047c71e4b0 service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:10.826199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931c054-3a7f-436e-8fe7-50047c71e4b0 service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:10.826530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931c054-3a7f-436e-8fe7-50047c71e4b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:10.828840 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-2931c054-3a7f-436e-8fe7-50047c71e4b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:10.841215 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931c054-3a7f-436e-8fe7-50047c71e4b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:10.842786 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931c054-3a7f-436e-8fe7-50047c71e4b0 service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:10.854075 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1248/2471] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:06:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:06:11.265072 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cbb88-7ecb-4a02-81a6-6d84859ca0ee service nova] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:11.274698 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cbb88-7ecb-4a02-81a6-6d84859ca0ee service nova] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:11.278737 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2904db-d201-4f51-a8e6-b7cb2851c915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:11.279341 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2904db-d201-4f51-a8e6-b7cb2851c915 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:11.279620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2904db-d201-4f51-a8e6-b7cb2851c915 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:11.279892 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2904db-d201-4f51-a8e6-b7cb2851c915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:11.282580 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-cf2904db-d201-4f51-a8e6-b7cb2851c915 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:11.348484 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2904db-d201-4f51-a8e6-b7cb2851c915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:11.349943 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2904db-d201-4f51-a8e6-b7cb2851c915 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:11.360433 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1224/2472] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:06:14.121082 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931c054-3a7f-436e-8fe7-50047c71e4b0 service nova] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:14.132051 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931c054-3a7f-436e-8fe7-50047c71e4b0 service nova] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:14.136182 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b15da1-5022-4a02-bb0a-2aed0df072fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:14.136865 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b15da1-5022-4a02-bb0a-2aed0df072fe service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:14.137211 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b15da1-5022-4a02-bb0a-2aed0df072fe service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:14.137548 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b15da1-5022-4a02-bb0a-2aed0df072fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:14.147158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b15da1-5022-4a02-bb0a-2aed0df072fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:14.148622 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b15da1-5022-4a02-bb0a-2aed0df072fe service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:14.158799 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1249/2473] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:06:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:06:14.818871 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2904db-d201-4f51-a8e6-b7cb2851c915 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:14.821066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-cf2904db-d201-4f51-a8e6-b7cb2851c915 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:14.835722 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2904db-d201-4f51-a8e6-b7cb2851c915 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:14.841217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b389e-47f1-415a-a9c8-dc1bf1a7411f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:14.841868 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b389e-47f1-415a-a9c8-dc1bf1a7411f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:14.842148 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b389e-47f1-415a-a9c8-dc1bf1a7411f admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:14.842424 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b389e-47f1-415a-a9c8-dc1bf1a7411f admin admin] PATH_INFO: `/v3/users/7353722ccb7548a6b92c89d4917fefaa` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:14.843718 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b389e-47f1-415a-a9c8-dc1bf1a7411f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7353722ccb7548a6b92c89d4917fefaa)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:14.845692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b389e-47f1-415a-a9c8-dc1bf1a7411f admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:14.901492 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-721b389e-47f1-415a-a9c8-dc1bf1a7411f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7353722ccb7548a6b92c89d4917fefaa'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:14.902157 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-721b389e-47f1-415a-a9c8-dc1bf1a7411f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7353722ccb7548a6b92c89d4917fefaa'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:14.916377 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-721b389e-47f1-415a-a9c8-dc1bf1a7411f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7353722ccb7548a6b92c89d4917fefaa'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:14.925207 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-721b389e-47f1-415a-a9c8-dc1bf1a7411f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7353722ccb7548a6b92c89d4917fefaa'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:14.926084 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-721b389e-47f1-415a-a9c8-dc1bf1a7411f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7353722ccb7548a6b92c89d4917fefaa'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:14.939976 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1225/2474] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 01:06:14 2022] DELETE /identity/v3/users/7353722ccb7548a6b92c89d4917fefaa => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:15.122286 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b15da1-5022-4a02-bb0a-2aed0df072fe service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:15.135134 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b15da1-5022-4a02-bb0a-2aed0df072fe service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:15.208833 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babe124-8c8e-49c9-8a3d-2c71950f3bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:15.210148 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babe124-8c8e-49c9-8a3d-2c71950f3bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:15.210427 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babe124-8c8e-49c9-8a3d-2c71950f3bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:15.210701 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babe124-8c8e-49c9-8a3d-2c71950f3bfc admin admin] PATH_INFO: `/v3/projects/dc70ef555fc84a51b3521007d00b77e3` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:15.211849 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1babe124-8c8e-49c9-8a3d-2c71950f3bfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc70ef555fc84a51b3521007d00b77e3)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:15.213795 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1babe124-8c8e-49c9-8a3d-2c71950f3bfc admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:15.277414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-1babe124-8c8e-49c9-8a3d-2c71950f3bfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc70ef555fc84a51b3521007d00b77e3', 'request_id': 'req-1babe124-8c8e-49c9-8a3d-2c71950f3bfc'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:15.285961 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-1babe124-8c8e-49c9-8a3d-2c71950f3bfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc70ef555fc84a51b3521007d00b77e3', 'request_id': 'req-1babe124-8c8e-49c9-8a3d-2c71950f3bfc'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:15.294183 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-1babe124-8c8e-49c9-8a3d-2c71950f3bfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc70ef555fc84a51b3521007d00b77e3', 'request_id': 'req-1babe124-8c8e-49c9-8a3d-2c71950f3bfc'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:15.307031 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-1babe124-8c8e-49c9-8a3d-2c71950f3bfc admin admin] The token cache is being invalidate because project dc70ef555fc84a51b3521007d00b77e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:06:15.307336 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-1babe124-8c8e-49c9-8a3d-2c71950f3bfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:15.307952 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-1babe124-8c8e-49c9-8a3d-2c71950f3bfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:15.309500 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1250/2475] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 01:06:15 2022] DELETE /identity/v3/projects/dc70ef555fc84a51b3521007d00b77e3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:15.316800 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91024d30-d21e-444c-b283-7efee9e23c7c None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:15.317417 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91024d30-d21e-444c-b283-7efee9e23c7c None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:15.317900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91024d30-d21e-444c-b283-7efee9e23c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:15.339873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-91024d30-d21e-444c-b283-7efee9e23c7c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:15.341419 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-91024d30-d21e-444c-b283-7efee9e23c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:15.424123 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1226/2476] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:06:15.430749 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8087c-fb94-4b2e-b1d7-5c0a904f9bf5 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:15.431119 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8087c-fb94-4b2e-b1d7-5c0a904f9bf5 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:15.431464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8087c-fb94-4b2e-b1d7-5c0a904f9bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:15.463758 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-ebc8087c-fb94-4b2e-b1d7-5c0a904f9bf5 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:15.465123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ebc8087c-fb94-4b2e-b1d7-5c0a904f9bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:15.474409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1251/2477] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:06:15.480448 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91024d30-d21e-444c-b283-7efee9e23c7c None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:15.494491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91024d30-d21e-444c-b283-7efee9e23c7c None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:15.501165 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef1feb-0783-4525-8cc1-330b2523d354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:15.503182 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef1feb-0783-4525-8cc1-330b2523d354 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:15.503490 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef1feb-0783-4525-8cc1-330b2523d354 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:15.503910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef1feb-0783-4525-8cc1-330b2523d354 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:15.504626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef1feb-0783-4525-8cc1-330b2523d354 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:06:15.504991 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef1feb-0783-4525-8cc1-330b2523d354 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:15.506656 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef1feb-0783-4525-8cc1-330b2523d354 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:15.508481 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:15.523116 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1227/2478] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 01:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:06:15.527696 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8087c-fb94-4b2e-b1d7-5c0a904f9bf5 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:15.539477 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8087c-fb94-4b2e-b1d7-5c0a904f9bf5 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:15.547492 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86c8c2-6ec1-4004-ac70-0523224bdc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:15.548143 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86c8c2-6ec1-4004-ac70-0523224bdc34 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:15.548425 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86c8c2-6ec1-4004-ac70-0523224bdc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:15.548697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86c8c2-6ec1-4004-ac70-0523224bdc34 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:15.549337 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86c8c2-6ec1-4004-ac70-0523224bdc34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:15.551266 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86c8c2-6ec1-4004-ac70-0523224bdc34 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:15.586964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1252/2479] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 01:06:15 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:06:15.588229 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef1feb-0783-4525-8cc1-330b2523d354 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:15.598856 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef1feb-0783-4525-8cc1-330b2523d354 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:15.669156 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e908520-2a9c-4120-bed8-4fdd3db06718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:15.669772 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e908520-2a9c-4120-bed8-4fdd3db06718 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:15.670050 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e908520-2a9c-4120-bed8-4fdd3db06718 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:15.670503 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e908520-2a9c-4120-bed8-4fdd3db06718 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:15.672056 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e908520-2a9c-4120-bed8-4fdd3db06718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:15.674024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e908520-2a9c-4120-bed8-4fdd3db06718 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:15.703242 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1228/2480] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 01:06:15 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:06:15.707090 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86c8c2-6ec1-4004-ac70-0523224bdc34 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:15.724596 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86c8c2-6ec1-4004-ac70-0523224bdc34 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:15.729901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa31926-7eec-4d1a-8ab2-2da5fe19ade8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:15.730694 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa31926-7eec-4d1a-8ab2-2da5fe19ade8 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:15.731000 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa31926-7eec-4d1a-8ab2-2da5fe19ade8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:15.731344 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa31926-7eec-4d1a-8ab2-2da5fe19ade8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:15.732312 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa31926-7eec-4d1a-8ab2-2da5fe19ade8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:15.735834 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa31926-7eec-4d1a-8ab2-2da5fe19ade8 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:15.750778 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1253/2481] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:06:15.752182 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e908520-2a9c-4120-bed8-4fdd3db06718 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:15.765234 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e908520-2a9c-4120-bed8-4fdd3db06718 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:15.770006 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b0c12-645a-4c5c-90a7-835e52fbb0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:15.770612 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b0c12-645a-4c5c-90a7-835e52fbb0c5 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:15.770890 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b0c12-645a-4c5c-90a7-835e52fbb0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:15.771166 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b0c12-645a-4c5c-90a7-835e52fbb0c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:15.771812 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b0c12-645a-4c5c-90a7-835e52fbb0c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:15.773453 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b0c12-645a-4c5c-90a7-835e52fbb0c5 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:15.786407 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1229/2482] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:06:15.790691 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa31926-7eec-4d1a-8ab2-2da5fe19ade8 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:15.801944 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa31926-7eec-4d1a-8ab2-2da5fe19ade8 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:15.806771 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202b471-7fcb-4d97-a306-d3b1a75064ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:15.807437 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202b471-7fcb-4d97-a306-d3b1a75064ef admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:15.807720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202b471-7fcb-4d97-a306-d3b1a75064ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:15.807999 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202b471-7fcb-4d97-a306-d3b1a75064ef admin admin] PATH_INFO: `/v3/projects/fb6bef9b9f36443cbdd1a3d2e4058eca/users/e61e7c203eff4d6bb5b5b35de8bafd09/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:15.823754 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202b471-7fcb-4d97-a306-d3b1a75064ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb6bef9b9f36443cbdd1a3d2e4058eca, user_id=e61e7c203eff4d6bb5b5b35de8bafd09, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:15.826468 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202b471-7fcb-4d97-a306-d3b1a75064ef admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:15.837286 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:15.842868 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1254/2483] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:06:15 2022] PUT /identity/v3/projects/fb6bef9b9f36443cbdd1a3d2e4058eca/users/e61e7c203eff4d6bb5b5b35de8bafd09/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:06:15.854442 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2edaa-c514-4f3d-8d83-be0256144f1b None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:15.855003 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2edaa-c514-4f3d-8d83-be0256144f1b None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:15.855109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2edaa-c514-4f3d-8d83-be0256144f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:15.887356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-76a2edaa-c514-4f3d-8d83-be0256144f1b None None] MFA Rules not processed for user `e61e7c203eff4d6bb5b5b35de8bafd09`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:15.888569 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-76a2edaa-c514-4f3d-8d83-be0256144f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:15.983027 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1230/2484] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:06:15.996528 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202b471-7fcb-4d97-a306-d3b1a75064ef admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:15.998317 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-8202b471-7fcb-4d97-a306-d3b1a75064ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:16.084226 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202b471-7fcb-4d97-a306-d3b1a75064ef admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:16.090373 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1d85e-a872-48d2-949a-8e238b52d1df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:16.090997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1d85e-a872-48d2-949a-8e238b52d1df service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:16.091287 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1d85e-a872-48d2-949a-8e238b52d1df service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:16.091555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1d85e-a872-48d2-949a-8e238b52d1df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:16.100986 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1d85e-a872-48d2-949a-8e238b52d1df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:16.102738 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1d85e-a872-48d2-949a-8e238b52d1df service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:16.112661 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1255/2485] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:06:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:06:24.684907 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2edaa-c514-4f3d-8d83-be0256144f1b None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:24.747246 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:24.791939 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2edaa-c514-4f3d-8d83-be0256144f1b None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:24.798074 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f2d73-6386-4e3b-9c8f-2402cc259283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:24.798809 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f2d73-6386-4e3b-9c8f-2402cc259283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:24.799151 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f2d73-6386-4e3b-9c8f-2402cc259283 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:24.799432 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f2d73-6386-4e3b-9c8f-2402cc259283 admin admin] PATH_INFO: `/v3/users/e61e7c203eff4d6bb5b5b35de8bafd09` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:24.800681 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f2d73-6386-4e3b-9c8f-2402cc259283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e61e7c203eff4d6bb5b5b35de8bafd09)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:24.802705 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f2d73-6386-4e3b-9c8f-2402cc259283 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:24.852068 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f10f2d73-6386-4e3b-9c8f-2402cc259283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e61e7c203eff4d6bb5b5b35de8bafd09'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:24.852748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f10f2d73-6386-4e3b-9c8f-2402cc259283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e61e7c203eff4d6bb5b5b35de8bafd09'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:24.864602 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f10f2d73-6386-4e3b-9c8f-2402cc259283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e61e7c203eff4d6bb5b5b35de8bafd09'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:24.871371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f10f2d73-6386-4e3b-9c8f-2402cc259283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e61e7c203eff4d6bb5b5b35de8bafd09'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:24.872043 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-f10f2d73-6386-4e3b-9c8f-2402cc259283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e61e7c203eff4d6bb5b5b35de8bafd09'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:24.886992 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1231/2486] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 01:06:24 2022] DELETE /identity/v3/users/e61e7c203eff4d6bb5b5b35de8bafd09 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:24.891935 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921bbf2-2908-4e12-a991-e1b967421236 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:24.892463 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921bbf2-2908-4e12-a991-e1b967421236 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:24.892576 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921bbf2-2908-4e12-a991-e1b967421236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:24.943775 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-b921bbf2-2908-4e12-a991-e1b967421236 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:24.944970 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-b921bbf2-2908-4e12-a991-e1b967421236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:25.005052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1256/2487] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:06:25.014741 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f2d73-6386-4e3b-9c8f-2402cc259283 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:25.016192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-f10f2d73-6386-4e3b-9c8f-2402cc259283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:25.090507 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f2d73-6386-4e3b-9c8f-2402cc259283 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:25.094258 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1da9c9-187c-4213-83a7-bc0ad2362a18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:25.094858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1da9c9-187c-4213-83a7-bc0ad2362a18 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:25.095142 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1da9c9-187c-4213-83a7-bc0ad2362a18 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:25.095417 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1da9c9-187c-4213-83a7-bc0ad2362a18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:25.105818 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1da9c9-187c-4213-83a7-bc0ad2362a18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:25.107264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1da9c9-187c-4213-83a7-bc0ad2362a18 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:25.117541 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1232/2488] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:06:25.392059 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921bbf2-2908-4e12-a991-e1b967421236 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:25.393771 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-b921bbf2-2908-4e12-a991-e1b967421236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:25.407280 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921bbf2-2908-4e12-a991-e1b967421236 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:25.412926 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb7a0c-1475-4a3d-a421-12fc36e1a4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:25.413649 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb7a0c-1475-4a3d-a421-12fc36e1a4e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:25.413993 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb7a0c-1475-4a3d-a421-12fc36e1a4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:25.414357 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb7a0c-1475-4a3d-a421-12fc36e1a4e5 admin admin] PATH_INFO: `/v3/projects/fb6bef9b9f36443cbdd1a3d2e4058eca` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:25.415697 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb7a0c-1475-4a3d-a421-12fc36e1a4e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb6bef9b9f36443cbdd1a3d2e4058eca)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:25.418126 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb7a0c-1475-4a3d-a421-12fc36e1a4e5 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:25.471142 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-94cb7a0c-1475-4a3d-a421-12fc36e1a4e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb6bef9b9f36443cbdd1a3d2e4058eca', 'request_id': 'req-94cb7a0c-1475-4a3d-a421-12fc36e1a4e5'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:25.475019 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:25.479909 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-94cb7a0c-1475-4a3d-a421-12fc36e1a4e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb6bef9b9f36443cbdd1a3d2e4058eca', 'request_id': 'req-94cb7a0c-1475-4a3d-a421-12fc36e1a4e5'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:25.487375 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-94cb7a0c-1475-4a3d-a421-12fc36e1a4e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb6bef9b9f36443cbdd1a3d2e4058eca', 'request_id': 'req-94cb7a0c-1475-4a3d-a421-12fc36e1a4e5'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:25.498379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-94cb7a0c-1475-4a3d-a421-12fc36e1a4e5 admin admin] The token cache is being invalidate because project fb6bef9b9f36443cbdd1a3d2e4058eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:06:25.498665 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-94cb7a0c-1475-4a3d-a421-12fc36e1a4e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:25.499455 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-94cb7a0c-1475-4a3d-a421-12fc36e1a4e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:25.500952 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1257/2489] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 01:06:25 2022] DELETE /identity/v3/projects/fb6bef9b9f36443cbdd1a3d2e4058eca => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:25.506184 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f204b-93f4-40da-b8de-e50d927eddab None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:25.506470 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f204b-93f4-40da-b8de-e50d927eddab None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:25.506744 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f204b-93f4-40da-b8de-e50d927eddab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:25.545445 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:25.558863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-e71f204b-93f4-40da-b8de-e50d927eddab None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:25.560066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e71f204b-93f4-40da-b8de-e50d927eddab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:25.618900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1233/2490] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:25.628029 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e85a3-d12c-40b2-8c12-7b0fd2e59e16 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:25.628343 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e85a3-d12c-40b2-8c12-7b0fd2e59e16 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:25.628618 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e85a3-d12c-40b2-8c12-7b0fd2e59e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:25.648936 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-d91e85a3-d12c-40b2-8c12-7b0fd2e59e16 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:25.650143 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d91e85a3-d12c-40b2-8c12-7b0fd2e59e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:25.658602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1258/2491] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:25.662699 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f204b-93f4-40da-b8de-e50d927eddab None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:25.672839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f204b-93f4-40da-b8de-e50d927eddab None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:25.677341 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3528c-be68-489f-875e-dd372ff44557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:25.677928 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3528c-be68-489f-875e-dd372ff44557 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:25.678203 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3528c-be68-489f-875e-dd372ff44557 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:25.679072 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3528c-be68-489f-875e-dd372ff44557 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:25.679722 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3528c-be68-489f-875e-dd372ff44557 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:06:25.680024 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3528c-be68-489f-875e-dd372ff44557 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:25.681657 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3528c-be68-489f-875e-dd372ff44557 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:25.694534 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1234/2492] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 01:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:06:25.699016 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e85a3-d12c-40b2-8c12-7b0fd2e59e16 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:25.710044 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e85a3-d12c-40b2-8c12-7b0fd2e59e16 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:25.714712 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ffdde-144a-4568-8864-9ccd9fc9a086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:25.715576 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ffdde-144a-4568-8864-9ccd9fc9a086 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:25.715856 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ffdde-144a-4568-8864-9ccd9fc9a086 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:25.716150 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ffdde-144a-4568-8864-9ccd9fc9a086 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:25.716806 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ffdde-144a-4568-8864-9ccd9fc9a086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:25.719450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ffdde-144a-4568-8864-9ccd9fc9a086 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:25.747551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1259/2493] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 01:06:25 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:06:25.751947 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3528c-be68-489f-875e-dd372ff44557 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:25.761824 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3528c-be68-489f-875e-dd372ff44557 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:25.819962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9215c77-4199-4166-a2a6-9fe3ad15efd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:25.820591 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9215c77-4199-4166-a2a6-9fe3ad15efd1 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:25.820875 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9215c77-4199-4166-a2a6-9fe3ad15efd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:25.821146 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9215c77-4199-4166-a2a6-9fe3ad15efd1 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:25.821778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9215c77-4199-4166-a2a6-9fe3ad15efd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:25.823949 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9215c77-4199-4166-a2a6-9fe3ad15efd1 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:25.846643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1235/2494] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 01:06:25 2022] POST /identity/v3/users => generated 303 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:06:25.850935 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ffdde-144a-4568-8864-9ccd9fc9a086 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:25.861874 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ffdde-144a-4568-8864-9ccd9fc9a086 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:25.866616 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6cb768-b16f-429b-9c7b-25c64bd0adb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:25.867215 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cb768-b16f-429b-9c7b-25c64bd0adb7 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:25.867497 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cb768-b16f-429b-9c7b-25c64bd0adb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:25.867772 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cb768-b16f-429b-9c7b-25c64bd0adb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:25.868476 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6cb768-b16f-429b-9c7b-25c64bd0adb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:25.870087 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6cb768-b16f-429b-9c7b-25c64bd0adb7 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:25.880612 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1260/2495] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:06:25.884564 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9215c77-4199-4166-a2a6-9fe3ad15efd1 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:25.895549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9215c77-4199-4166-a2a6-9fe3ad15efd1 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:25.900147 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007fddb-8dda-4da7-8107-2fdb9e945462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:25.900744 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007fddb-8dda-4da7-8107-2fdb9e945462 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:25.901022 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007fddb-8dda-4da7-8107-2fdb9e945462 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:25.901293 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007fddb-8dda-4da7-8107-2fdb9e945462 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:25.901927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007fddb-8dda-4da7-8107-2fdb9e945462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:25.903518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007fddb-8dda-4da7-8107-2fdb9e945462 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:25.915477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1236/2496] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:06:25.919742 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6cb768-b16f-429b-9c7b-25c64bd0adb7 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:25.929943 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6cb768-b16f-429b-9c7b-25c64bd0adb7 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:25.934857 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8eb36e-380a-4791-b187-80999b82bdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:25.935511 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8eb36e-380a-4791-b187-80999b82bdee admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:25.935792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8eb36e-380a-4791-b187-80999b82bdee admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:25.936064 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8eb36e-380a-4791-b187-80999b82bdee admin admin] PATH_INFO: `/v3/projects/12baabc7b5834df28474845bfb104895/users/1e365b508ecd4825ada3bf3bedccbf10/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:25.951892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8eb36e-380a-4791-b187-80999b82bdee admin admin] RBAC: Authorizing `identity:create_grant(project_id=12baabc7b5834df28474845bfb104895, user_id=1e365b508ecd4825ada3bf3bedccbf10, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:25.955634 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8eb36e-380a-4791-b187-80999b82bdee admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:25.967991 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1261/2497] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:06:25 2022] PUT /identity/v3/projects/12baabc7b5834df28474845bfb104895/users/1e365b508ecd4825ada3bf3bedccbf10/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:25.973504 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94feeb-cc17-496e-9f3e-aaf3cf0a12ef None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:25.973895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94feeb-cc17-496e-9f3e-aaf3cf0a12ef None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:25.974232 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94feeb-cc17-496e-9f3e-aaf3cf0a12ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:25.993004 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-7e94feeb-cc17-496e-9f3e-aaf3cf0a12ef None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:25.994190 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-7e94feeb-cc17-496e-9f3e-aaf3cf0a12ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:26.052317 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1237/2498] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:26.062507 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8eb36e-380a-4791-b187-80999b82bdee admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:26.064312 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-1a8eb36e-380a-4791-b187-80999b82bdee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:26.121028 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8eb36e-380a-4791-b187-80999b82bdee admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:26.125212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee8f79-15b9-432e-9dfb-cc6e7f6b5970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:26.125890 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee8f79-15b9-432e-9dfb-cc6e7f6b5970 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:26.126171 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee8f79-15b9-432e-9dfb-cc6e7f6b5970 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:26.126442 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee8f79-15b9-432e-9dfb-cc6e7f6b5970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:26.136348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee8f79-15b9-432e-9dfb-cc6e7f6b5970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:26.137779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee8f79-15b9-432e-9dfb-cc6e7f6b5970 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:26.148330 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1262/2499] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:06:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:06:26.996337 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18748c9-d1a2-47e3-b3a6-afe8563ef5d5 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:26.996849 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18748c9-d1a2-47e3-b3a6-afe8563ef5d5 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:26.996949 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18748c9-d1a2-47e3-b3a6-afe8563ef5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:27.016583 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:27.027313 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-d18748c9-d1a2-47e3-b3a6-afe8563ef5d5 None None] MFA Rules not processed for user `1e365b508ecd4825ada3bf3bedccbf10`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:27.028594 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d18748c9-d1a2-47e3-b3a6-afe8563ef5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:27.099917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1238/2500] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:26 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:27.104533 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee8f79-15b9-432e-9dfb-cc6e7f6b5970 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:27.114807 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee8f79-15b9-432e-9dfb-cc6e7f6b5970 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:27.119379 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2eccf-8b8c-468a-bccb-1fbc299b8490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:27.120044 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2eccf-8b8c-468a-bccb-1fbc299b8490 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:27.120516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2eccf-8b8c-468a-bccb-1fbc299b8490 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:27.120851 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2eccf-8b8c-468a-bccb-1fbc299b8490 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:27.121547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2eccf-8b8c-468a-bccb-1fbc299b8490 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:27.123625 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2eccf-8b8c-468a-bccb-1fbc299b8490 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:27.156472 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1263/2501] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 01:06:27 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:06:27.157765 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18748c9-d1a2-47e3-b3a6-afe8563ef5d5 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:27.168273 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18748c9-d1a2-47e3-b3a6-afe8563ef5d5 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:27.223905 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a326c7-e2e5-407a-bc67-3ce946ec1dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:27.224540 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a326c7-e2e5-407a-bc67-3ce946ec1dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:27.224822 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a326c7-e2e5-407a-bc67-3ce946ec1dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:27.225100 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a326c7-e2e5-407a-bc67-3ce946ec1dc9 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:27.225735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a326c7-e2e5-407a-bc67-3ce946ec1dc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:27.227693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a326c7-e2e5-407a-bc67-3ce946ec1dc9 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:27.248405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1239/2502] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 01:06:27 2022] POST /identity/v3/users => generated 308 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:06:27.253156 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2eccf-8b8c-468a-bccb-1fbc299b8490 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:27.263153 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2eccf-8b8c-468a-bccb-1fbc299b8490 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:27.269354 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab91f6-cf41-4344-bca3-8715c941670b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:27.269965 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab91f6-cf41-4344-bca3-8715c941670b admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:27.270230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab91f6-cf41-4344-bca3-8715c941670b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:27.270504 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab91f6-cf41-4344-bca3-8715c941670b admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:27.271137 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab91f6-cf41-4344-bca3-8715c941670b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:27.272756 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab91f6-cf41-4344-bca3-8715c941670b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:27.283824 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1264/2503] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:06:27.287868 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a326c7-e2e5-407a-bc67-3ce946ec1dc9 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:27.297951 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a326c7-e2e5-407a-bc67-3ce946ec1dc9 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:27.302587 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994dcbc-75d9-41da-b058-65c424dd5bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:27.303235 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994dcbc-75d9-41da-b058-65c424dd5bc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:27.303520 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994dcbc-75d9-41da-b058-65c424dd5bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:27.303799 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994dcbc-75d9-41da-b058-65c424dd5bc2 admin admin] PATH_INFO: `/v3/projects/b3a6fafd2ea048eeae5da468b5f39651/users/71cd1f86ad80412cbb91340e4e0746d8/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:27.318553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994dcbc-75d9-41da-b058-65c424dd5bc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3a6fafd2ea048eeae5da468b5f39651, user_id=71cd1f86ad80412cbb91340e4e0746d8, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:27.321251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994dcbc-75d9-41da-b058-65c424dd5bc2 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:27.335989 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1240/2504] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:06:27 2022] PUT /identity/v3/projects/b3a6fafd2ea048eeae5da468b5f39651/users/71cd1f86ad80412cbb91340e4e0746d8/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:27.339894 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab91f6-cf41-4344-bca3-8715c941670b admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:27.350821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab91f6-cf41-4344-bca3-8715c941670b admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:27.394576 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:27.408040 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f452b4-794b-4203-957f-80a4b30c3c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:27.408672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f452b4-794b-4203-957f-80a4b30c3c4a admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:27.408955 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f452b4-794b-4203-957f-80a4b30c3c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:27.409229 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f452b4-794b-4203-957f-80a4b30c3c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:27.409859 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f452b4-794b-4203-957f-80a4b30c3c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:27.411449 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f452b4-794b-4203-957f-80a4b30c3c4a admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:27.421142 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1265/2505] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:06:27.425285 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994dcbc-75d9-41da-b058-65c424dd5bc2 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:27.435723 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994dcbc-75d9-41da-b058-65c424dd5bc2 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:27.441122 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192b230-794d-4ac2-9c3d-1c0fca16ea0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:27.441762 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192b230-794d-4ac2-9c3d-1c0fca16ea0a admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:27.442039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192b230-794d-4ac2-9c3d-1c0fca16ea0a admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:27.442314 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192b230-794d-4ac2-9c3d-1c0fca16ea0a admin admin] PATH_INFO: `/v3/projects/b3a6fafd2ea048eeae5da468b5f39651/users/71cd1f86ad80412cbb91340e4e0746d8/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:27.444240 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192b230-794d-4ac2-9c3d-1c0fca16ea0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3a6fafd2ea048eeae5da468b5f39651, user_id=71cd1f86ad80412cbb91340e4e0746d8, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:27.446876 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192b230-794d-4ac2-9c3d-1c0fca16ea0a admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:27.458195 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1241/2506] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:06:27 2022] PUT /identity/v3/projects/b3a6fafd2ea048eeae5da468b5f39651/users/71cd1f86ad80412cbb91340e4e0746d8/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:28.075609 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d43232-b99a-4b2f-9faf-afdc4c174f68 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:28.075903 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d43232-b99a-4b2f-9faf-afdc4c174f68 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:28.076231 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d43232-b99a-4b2f-9faf-afdc4c174f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:28.109061 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-b6d43232-b99a-4b2f-9faf-afdc4c174f68 None None] MFA Rules not processed for user `71cd1f86ad80412cbb91340e4e0746d8`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:28.110444 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-b6d43232-b99a-4b2f-9faf-afdc4c174f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:28.206692 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1266/2507] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:28 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:29.231730 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192b230-794d-4ac2-9c3d-1c0fca16ea0a admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:29.286678 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:29.299213 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192b230-794d-4ac2-9c3d-1c0fca16ea0a admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:29.303755 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a12175-64bb-4890-8d4f-d692ea2d94ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:29.304405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a12175-64bb-4890-8d4f-d692ea2d94ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:29.304683 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a12175-64bb-4890-8d4f-d692ea2d94ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:29.304955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a12175-64bb-4890-8d4f-d692ea2d94ae admin admin] PATH_INFO: `/v3/users/1e365b508ecd4825ada3bf3bedccbf10` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:29.306058 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a12175-64bb-4890-8d4f-d692ea2d94ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e365b508ecd4825ada3bf3bedccbf10)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:29.307962 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a12175-64bb-4890-8d4f-d692ea2d94ae admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:29.354175 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-44a12175-64bb-4890-8d4f-d692ea2d94ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e365b508ecd4825ada3bf3bedccbf10'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:29.354832 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-44a12175-64bb-4890-8d4f-d692ea2d94ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e365b508ecd4825ada3bf3bedccbf10'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:29.367418 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-44a12175-64bb-4890-8d4f-d692ea2d94ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e365b508ecd4825ada3bf3bedccbf10'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:29.374998 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-44a12175-64bb-4890-8d4f-d692ea2d94ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e365b508ecd4825ada3bf3bedccbf10'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:29.375673 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-44a12175-64bb-4890-8d4f-d692ea2d94ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e365b508ecd4825ada3bf3bedccbf10'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:29.388193 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1242/2508] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 01:06:29 2022] DELETE /identity/v3/users/1e365b508ecd4825ada3bf3bedccbf10 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:06:29.563864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d43232-b99a-4b2f-9faf-afdc4c174f68 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:29.574936 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d43232-b99a-4b2f-9faf-afdc4c174f68 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:29.635318 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0869872-0571-4ccc-966e-9d4c8c404482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:29.635951 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0869872-0571-4ccc-966e-9d4c8c404482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:29.636284 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0869872-0571-4ccc-966e-9d4c8c404482 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:29.636560 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0869872-0571-4ccc-966e-9d4c8c404482 admin admin] PATH_INFO: `/v3/projects/12baabc7b5834df28474845bfb104895` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:29.637680 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0869872-0571-4ccc-966e-9d4c8c404482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12baabc7b5834df28474845bfb104895)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:29.639737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0869872-0571-4ccc-966e-9d4c8c404482 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:29.687262 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-b0869872-0571-4ccc-966e-9d4c8c404482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12baabc7b5834df28474845bfb104895', 'request_id': 'req-b0869872-0571-4ccc-966e-9d4c8c404482'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:29.694643 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-b0869872-0571-4ccc-966e-9d4c8c404482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12baabc7b5834df28474845bfb104895', 'request_id': 'req-b0869872-0571-4ccc-966e-9d4c8c404482'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:29.700525 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-b0869872-0571-4ccc-966e-9d4c8c404482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12baabc7b5834df28474845bfb104895', 'request_id': 'req-b0869872-0571-4ccc-966e-9d4c8c404482'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:29.708104 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:29.712060 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-b0869872-0571-4ccc-966e-9d4c8c404482 admin admin] The token cache is being invalidate because project 12baabc7b5834df28474845bfb104895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:06:29.712478 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-b0869872-0571-4ccc-966e-9d4c8c404482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:29.713589 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-b0869872-0571-4ccc-966e-9d4c8c404482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:29.715279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1267/2509] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 01:06:29 2022] DELETE /identity/v3/projects/12baabc7b5834df28474845bfb104895 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:29.719867 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a12175-64bb-4890-8d4f-d692ea2d94ae admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:29.721348 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-44a12175-64bb-4890-8d4f-d692ea2d94ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:29.735179 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a12175-64bb-4890-8d4f-d692ea2d94ae admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:29.789858 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690a6b6-4198-4343-bd0b-f526cc2f520b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:29.790496 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690a6b6-4198-4343-bd0b-f526cc2f520b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:29.790780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690a6b6-4198-4343-bd0b-f526cc2f520b admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:29.791053 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690a6b6-4198-4343-bd0b-f526cc2f520b admin admin] PATH_INFO: `/v3/users/71cd1f86ad80412cbb91340e4e0746d8` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:29.792303 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690a6b6-4198-4343-bd0b-f526cc2f520b admin admin] RBAC: Authorizing `identity:delete_user(user_id=71cd1f86ad80412cbb91340e4e0746d8)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:29.794150 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690a6b6-4198-4343-bd0b-f526cc2f520b admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:29.841378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2690a6b6-4198-4343-bd0b-f526cc2f520b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71cd1f86ad80412cbb91340e4e0746d8'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:29.842004 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2690a6b6-4198-4343-bd0b-f526cc2f520b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71cd1f86ad80412cbb91340e4e0746d8'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:29.854459 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2690a6b6-4198-4343-bd0b-f526cc2f520b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71cd1f86ad80412cbb91340e4e0746d8'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:29.860415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:29.862546 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2690a6b6-4198-4343-bd0b-f526cc2f520b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71cd1f86ad80412cbb91340e4e0746d8'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:29.863481 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2690a6b6-4198-4343-bd0b-f526cc2f520b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71cd1f86ad80412cbb91340e4e0746d8'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:29.877505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1243/2510] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 01:06:29 2022] DELETE /identity/v3/users/71cd1f86ad80412cbb91340e4e0746d8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:30.049992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0869872-0571-4ccc-966e-9d4c8c404482 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:30.051463 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-b0869872-0571-4ccc-966e-9d4c8c404482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:30.064916 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0869872-0571-4ccc-966e-9d4c8c404482 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:30.121338 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609077b-b701-4447-9fdb-3a7c7eb52d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:30.121901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609077b-b701-4447-9fdb-3a7c7eb52d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:30.122181 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609077b-b701-4447-9fdb-3a7c7eb52d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:30.122453 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609077b-b701-4447-9fdb-3a7c7eb52d43 admin admin] PATH_INFO: `/v3/projects/b3a6fafd2ea048eeae5da468b5f39651` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:30.123563 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609077b-b701-4447-9fdb-3a7c7eb52d43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3a6fafd2ea048eeae5da468b5f39651)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:30.125714 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609077b-b701-4447-9fdb-3a7c7eb52d43 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:30.173143 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a609077b-b701-4447-9fdb-3a7c7eb52d43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3a6fafd2ea048eeae5da468b5f39651', 'request_id': 'req-a609077b-b701-4447-9fdb-3a7c7eb52d43'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:30.180470 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a609077b-b701-4447-9fdb-3a7c7eb52d43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3a6fafd2ea048eeae5da468b5f39651', 'request_id': 'req-a609077b-b701-4447-9fdb-3a7c7eb52d43'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:30.186428 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a609077b-b701-4447-9fdb-3a7c7eb52d43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3a6fafd2ea048eeae5da468b5f39651', 'request_id': 'req-a609077b-b701-4447-9fdb-3a7c7eb52d43'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:30.197814 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a609077b-b701-4447-9fdb-3a7c7eb52d43 admin admin] The token cache is being invalidate because project b3a6fafd2ea048eeae5da468b5f39651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:06:30.198100 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a609077b-b701-4447-9fdb-3a7c7eb52d43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:30.198726 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a609077b-b701-4447-9fdb-3a7c7eb52d43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:30.200275 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1268/2511] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 01:06:30 2022] DELETE /identity/v3/projects/b3a6fafd2ea048eeae5da468b5f39651 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:30.206012 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c06dc-d4a0-4842-a435-c731cf16dbc2 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:30.206300 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c06dc-d4a0-4842-a435-c731cf16dbc2 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:30.206595 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c06dc-d4a0-4842-a435-c731cf16dbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:30.226995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-0c2c06dc-d4a0-4842-a435-c731cf16dbc2 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:30.228236 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-0c2c06dc-d4a0-4842-a435-c731cf16dbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:30.287562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1244/2512] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:30.295436 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8f49c-952b-4cdd-8184-f67af70119c9 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:30.295785 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8f49c-952b-4cdd-8184-f67af70119c9 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:30.296064 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8f49c-952b-4cdd-8184-f67af70119c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:30.315913 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-9dc8f49c-952b-4cdd-8184-f67af70119c9 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:30.317146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-9dc8f49c-952b-4cdd-8184-f67af70119c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:30.325345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1269/2513] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:30.330058 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c06dc-d4a0-4842-a435-c731cf16dbc2 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:30.339767 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c06dc-d4a0-4842-a435-c731cf16dbc2 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:30.345027 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e5593-76fd-4497-a4d4-53e05ca95dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:30.345625 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e5593-76fd-4497-a4d4-53e05ca95dbc admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:30.345907 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e5593-76fd-4497-a4d4-53e05ca95dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:30.346181 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e5593-76fd-4497-a4d4-53e05ca95dbc admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:30.346822 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e5593-76fd-4497-a4d4-53e05ca95dbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:06:30.347125 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e5593-76fd-4497-a4d4-53e05ca95dbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:30.348782 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e5593-76fd-4497-a4d4-53e05ca95dbc admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:30.360822 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1245/2514] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 01:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:06:30.364925 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8f49c-952b-4cdd-8184-f67af70119c9 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:30.374174 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8f49c-952b-4cdd-8184-f67af70119c9 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:30.378868 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc153a0-ff2a-4dd8-8eaa-d0dfbc22ab8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:30.379457 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc153a0-ff2a-4dd8-8eaa-d0dfbc22ab8b admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:30.379736 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc153a0-ff2a-4dd8-8eaa-d0dfbc22ab8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:30.380008 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc153a0-ff2a-4dd8-8eaa-d0dfbc22ab8b admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:30.380665 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc153a0-ff2a-4dd8-8eaa-d0dfbc22ab8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:30.382556 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc153a0-ff2a-4dd8-8eaa-d0dfbc22ab8b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:30.410470 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1270/2515] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 01:06:30 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:06:30.414620 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e5593-76fd-4497-a4d4-53e05ca95dbc admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:30.424889 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e5593-76fd-4497-a4d4-53e05ca95dbc admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:30.482171 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafe418-e4d3-4d2d-8203-0f613444f3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:30.482779 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafe418-e4d3-4d2d-8203-0f613444f3df admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:30.483064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafe418-e4d3-4d2d-8203-0f613444f3df admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:30.483344 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafe418-e4d3-4d2d-8203-0f613444f3df admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:30.483981 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafe418-e4d3-4d2d-8203-0f613444f3df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:30.485934 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafe418-e4d3-4d2d-8203-0f613444f3df admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:30.507445 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1246/2516] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 01:06:30 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:06:30.511582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc153a0-ff2a-4dd8-8eaa-d0dfbc22ab8b admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:30.522375 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc153a0-ff2a-4dd8-8eaa-d0dfbc22ab8b admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:30.526959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2246f-0c21-4970-b7bc-8aaa4c547232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:30.527547 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2246f-0c21-4970-b7bc-8aaa4c547232 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:30.527822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2246f-0c21-4970-b7bc-8aaa4c547232 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:30.528110 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2246f-0c21-4970-b7bc-8aaa4c547232 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:30.528742 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2246f-0c21-4970-b7bc-8aaa4c547232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:30.530399 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2246f-0c21-4970-b7bc-8aaa4c547232 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:30.540232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1271/2517] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:06:30.544661 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafe418-e4d3-4d2d-8203-0f613444f3df admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:30.556146 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafe418-e4d3-4d2d-8203-0f613444f3df admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:30.560816 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11642c5d-71de-414c-8ba5-ef2bb507c7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:30.562033 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11642c5d-71de-414c-8ba5-ef2bb507c7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:30.562321 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11642c5d-71de-414c-8ba5-ef2bb507c7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:30.562602 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11642c5d-71de-414c-8ba5-ef2bb507c7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:30.563243 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11642c5d-71de-414c-8ba5-ef2bb507c7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:30.565177 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11642c5d-71de-414c-8ba5-ef2bb507c7a9 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:30.576453 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1247/2518] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:06:30.581106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2246f-0c21-4970-b7bc-8aaa4c547232 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:30.591004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2246f-0c21-4970-b7bc-8aaa4c547232 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:30.596059 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc086cc5-435d-4657-8ff0-9aa5399a1426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:30.596724 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc086cc5-435d-4657-8ff0-9aa5399a1426 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:30.597006 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc086cc5-435d-4657-8ff0-9aa5399a1426 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:30.597284 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc086cc5-435d-4657-8ff0-9aa5399a1426 admin admin] PATH_INFO: `/v3/projects/cf7545997eb64ce7897101fac680fd7c/users/f3df72de1a3040c1a20aa88e0aa18c3b/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:30.611688 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc086cc5-435d-4657-8ff0-9aa5399a1426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf7545997eb64ce7897101fac680fd7c, user_id=f3df72de1a3040c1a20aa88e0aa18c3b, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:30.614364 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc086cc5-435d-4657-8ff0-9aa5399a1426 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:30.630212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1272/2519] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:06:30 2022] PUT /identity/v3/projects/cf7545997eb64ce7897101fac680fd7c/users/f3df72de1a3040c1a20aa88e0aa18c3b/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:30.641754 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd08f68-e863-4e6b-97bf-6e64e4c91057 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:30.642038 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd08f68-e863-4e6b-97bf-6e64e4c91057 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:30.642311 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd08f68-e863-4e6b-97bf-6e64e4c91057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:30.669719 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-2cd08f68-e863-4e6b-97bf-6e64e4c91057 None None] MFA Rules not processed for user `f3df72de1a3040c1a20aa88e0aa18c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:30.671446 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-2cd08f68-e863-4e6b-97bf-6e64e4c91057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:30.718534 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:30.742327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1248/2520] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:06:30.746718 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc086cc5-435d-4657-8ff0-9aa5399a1426 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:30.777737 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:30.811275 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc086cc5-435d-4657-8ff0-9aa5399a1426 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:30.816976 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28bca9-ce38-4acb-b016-e5e9b8618111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:30.817590 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28bca9-ce38-4acb-b016-e5e9b8618111 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:30.817869 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28bca9-ce38-4acb-b016-e5e9b8618111 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:30.818141 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28bca9-ce38-4acb-b016-e5e9b8618111 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:30.818766 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c28bca9-ce38-4acb-b016-e5e9b8618111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:30.820764 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c28bca9-ce38-4acb-b016-e5e9b8618111 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:30.846545 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1273/2521] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 01:06:30 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:06:30.850509 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd08f68-e863-4e6b-97bf-6e64e4c91057 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:30.861570 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd08f68-e863-4e6b-97bf-6e64e4c91057 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:30.919501 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65961f-8e3e-4847-b52c-3be2c1d48acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:30.920144 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65961f-8e3e-4847-b52c-3be2c1d48acd admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:30.920429 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65961f-8e3e-4847-b52c-3be2c1d48acd admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:30.921228 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65961f-8e3e-4847-b52c-3be2c1d48acd admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:30.921897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65961f-8e3e-4847-b52c-3be2c1d48acd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:30.923805 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65961f-8e3e-4847-b52c-3be2c1d48acd admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:30.947757 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1249/2522] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 01:06:30 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:06:30.950735 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28bca9-ce38-4acb-b016-e5e9b8618111 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:30.964209 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28bca9-ce38-4acb-b016-e5e9b8618111 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:30.969219 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2ccdf-3b00-49b6-94c0-7733b2a54521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:30.969820 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2ccdf-3b00-49b6-94c0-7733b2a54521 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:30.970104 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2ccdf-3b00-49b6-94c0-7733b2a54521 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:30.970378 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2ccdf-3b00-49b6-94c0-7733b2a54521 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:30.971011 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2ccdf-3b00-49b6-94c0-7733b2a54521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:30.972641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2ccdf-3b00-49b6-94c0-7733b2a54521 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:30.985212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1274/2523] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:06:30.989309 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65961f-8e3e-4847-b52c-3be2c1d48acd admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:31.000891 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65961f-8e3e-4847-b52c-3be2c1d48acd admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:31.006066 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b189d37-a666-4bc8-ada0-3a4959c008c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:31.007071 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b189d37-a666-4bc8-ada0-3a4959c008c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:31.007355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b189d37-a666-4bc8-ada0-3a4959c008c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:31.007626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b189d37-a666-4bc8-ada0-3a4959c008c5 admin admin] PATH_INFO: `/v3/projects/ae759d31f45249c09b4a8dd633570cbc/users/b3c4334f2cf4469a8cf04b89f8398e54/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:31.023005 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b189d37-a666-4bc8-ada0-3a4959c008c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae759d31f45249c09b4a8dd633570cbc, user_id=b3c4334f2cf4469a8cf04b89f8398e54, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:31.025692 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b189d37-a666-4bc8-ada0-3a4959c008c5 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:31.038952 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1250/2524] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:06:30 2022] PUT /identity/v3/projects/ae759d31f45249c09b4a8dd633570cbc/users/b3c4334f2cf4469a8cf04b89f8398e54/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:31.047741 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2ccdf-3b00-49b6-94c0-7733b2a54521 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:31.057687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2ccdf-3b00-49b6-94c0-7733b2a54521 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:31.114679 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b2763-edd5-4444-a3db-e7a8ca76a71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:31.115290 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b2763-edd5-4444-a3db-e7a8ca76a71f admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:31.115565 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b2763-edd5-4444-a3db-e7a8ca76a71f admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:31.115835 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b2763-edd5-4444-a3db-e7a8ca76a71f admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:31.116494 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b2763-edd5-4444-a3db-e7a8ca76a71f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:31.118071 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b2763-edd5-4444-a3db-e7a8ca76a71f admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:31.128797 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1275/2525] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:06:31.133148 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b189d37-a666-4bc8-ada0-3a4959c008c5 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:31.143377 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b189d37-a666-4bc8-ada0-3a4959c008c5 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:31.148069 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb0892-30ee-4a73-8d24-2293b9a39bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:31.148799 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb0892-30ee-4a73-8d24-2293b9a39bd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:31.149078 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb0892-30ee-4a73-8d24-2293b9a39bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:31.149346 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb0892-30ee-4a73-8d24-2293b9a39bd8 admin admin] PATH_INFO: `/v3/projects/ae759d31f45249c09b4a8dd633570cbc/users/b3c4334f2cf4469a8cf04b89f8398e54/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:31.151263 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb0892-30ee-4a73-8d24-2293b9a39bd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae759d31f45249c09b4a8dd633570cbc, user_id=b3c4334f2cf4469a8cf04b89f8398e54, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:31.153910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb0892-30ee-4a73-8d24-2293b9a39bd8 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:31.164431 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1251/2526] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:06:31 2022] PUT /identity/v3/projects/ae759d31f45249c09b4a8dd633570cbc/users/b3c4334f2cf4469a8cf04b89f8398e54/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:06:31.176864 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55765f08-51a5-486f-9cd6-348035a680ed None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:31.177074 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55765f08-51a5-486f-9cd6-348035a680ed None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:31.177345 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55765f08-51a5-486f-9cd6-348035a680ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:31.206300 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-55765f08-51a5-486f-9cd6-348035a680ed None None] MFA Rules not processed for user `b3c4334f2cf4469a8cf04b89f8398e54`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:31.207481 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-55765f08-51a5-486f-9cd6-348035a680ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:31.275955 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1276/2527] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:31 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:31.341489 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb0892-30ee-4a73-8d24-2293b9a39bd8 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:31.343947 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-95bb0892-30ee-4a73-8d24-2293b9a39bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:31.401610 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb0892-30ee-4a73-8d24-2293b9a39bd8 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:31.406207 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138ac23-8446-43ec-b9be-0ce70773f954 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:31.406811 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138ac23-8446-43ec-b9be-0ce70773f954 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:31.407089 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138ac23-8446-43ec-b9be-0ce70773f954 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:31.407360 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138ac23-8446-43ec-b9be-0ce70773f954 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:31.416074 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138ac23-8446-43ec-b9be-0ce70773f954 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:31.417542 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138ac23-8446-43ec-b9be-0ce70773f954 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:31.427441 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1252/2528] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:06:31 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:06:31.734845 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55765f08-51a5-486f-9cd6-348035a680ed None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:31.743955 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55765f08-51a5-486f-9cd6-348035a680ed None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:31.748776 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201c64b-22b8-49ab-9843-37bcaf7d0534 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c4334f2cf4469a8cf04b89f8398e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae759d31f45249c09b4a8dd633570cbc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:31.749381 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201c64b-22b8-49ab-9843-37bcaf7d0534 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:31.749671 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201c64b-22b8-49ab-9843-37bcaf7d0534 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:31.749951 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201c64b-22b8-49ab-9843-37bcaf7d0534 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:31.750602 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201c64b-22b8-49ab-9843-37bcaf7d0534 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:06:31.750908 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201c64b-22b8-49ab-9843-37bcaf7d0534 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:31.752583 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201c64b-22b8-49ab-9843-37bcaf7d0534 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:31.765309 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1277/2529] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 01:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:06:31.768739 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138ac23-8446-43ec-b9be-0ce70773f954 service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:31.778362 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138ac23-8446-43ec-b9be-0ce70773f954 service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:31.783665 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c11c31-de87-4521-9910-822c155bca9d tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c4334f2cf4469a8cf04b89f8398e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae759d31f45249c09b4a8dd633570cbc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:31.784291 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c11c31-de87-4521-9910-822c155bca9d tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:31.784596 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c11c31-de87-4521-9910-822c155bca9d tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:31.784868 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c11c31-de87-4521-9910-822c155bca9d tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:31.785500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c11c31-de87-4521-9910-822c155bca9d tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:31.787704 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c11c31-de87-4521-9910-822c155bca9d tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:31.814012 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1253/2530] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 01:06:31 2022] POST /identity/v3/projects => generated 353 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:06:31.818109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201c64b-22b8-49ab-9843-37bcaf7d0534 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:31.829094 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201c64b-22b8-49ab-9843-37bcaf7d0534 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:31.833707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:31.886860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a3b17-4909-4077-b1ee-8de106d9b9a4 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c4334f2cf4469a8cf04b89f8398e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae759d31f45249c09b4a8dd633570cbc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:31.887466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a3b17-4909-4077-b1ee-8de106d9b9a4 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:31.887748 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a3b17-4909-4077-b1ee-8de106d9b9a4 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:31.888021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a3b17-4909-4077-b1ee-8de106d9b9a4 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:31.888725 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a3b17-4909-4077-b1ee-8de106d9b9a4 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:06:31.889030 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a3b17-4909-4077-b1ee-8de106d9b9a4 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:31.891326 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a3b17-4909-4077-b1ee-8de106d9b9a4 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:31.904615 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1278/2531] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 01:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 30 01:06:31.943077 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c11c31-de87-4521-9910-822c155bca9d tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:31.954287 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c11c31-de87-4521-9910-822c155bca9d tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:31.958796 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1e85d-3f99-4eb7-857b-e1dd8d9bf7ea tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3c4334f2cf4469a8cf04b89f8398e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae759d31f45249c09b4a8dd633570cbc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:31.959419 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1e85d-3f99-4eb7-857b-e1dd8d9bf7ea tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:31.959702 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1e85d-3f99-4eb7-857b-e1dd8d9bf7ea tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:31.959979 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1e85d-3f99-4eb7-857b-e1dd8d9bf7ea tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] PATH_INFO: `/v3/projects/6a7166f324ac4171923bfbf4a4791e2d` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:31.961127 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1e85d-3f99-4eb7-857b-e1dd8d9bf7ea tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6a7166f324ac4171923bfbf4a4791e2d)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:31.963456 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1e85d-3f99-4eb7-857b-e1dd8d9bf7ea tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:31.976376 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:32.012513 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-91e1e85d-3f99-4eb7-857b-e1dd8d9bf7ea tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a7166f324ac4171923bfbf4a4791e2d', 'request_id': 'req-91e1e85d-3f99-4eb7-857b-e1dd8d9bf7ea'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:32.021735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-91e1e85d-3f99-4eb7-857b-e1dd8d9bf7ea tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a7166f324ac4171923bfbf4a4791e2d', 'request_id': 'req-91e1e85d-3f99-4eb7-857b-e1dd8d9bf7ea'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:32.029377 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-91e1e85d-3f99-4eb7-857b-e1dd8d9bf7ea tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a7166f324ac4171923bfbf4a4791e2d', 'request_id': 'req-91e1e85d-3f99-4eb7-857b-e1dd8d9bf7ea'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:32.039733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-91e1e85d-3f99-4eb7-857b-e1dd8d9bf7ea tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] The token cache is being invalidate because project 6a7166f324ac4171923bfbf4a4791e2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:06:32.040027 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-91e1e85d-3f99-4eb7-857b-e1dd8d9bf7ea tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:32.040795 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-91e1e85d-3f99-4eb7-857b-e1dd8d9bf7ea tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:32.042375 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1254/2532] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 01:06:31 2022] DELETE /identity/v3/projects/6a7166f324ac4171923bfbf4a4791e2d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:32.230705 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a3b17-4909-4077-b1ee-8de106d9b9a4 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:32.232500 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a48a3b17-4909-4077-b1ee-8de106d9b9a4 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:32.300855 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a3b17-4909-4077-b1ee-8de106d9b9a4 tempest-VolumeQuotaClassesTest-1989696018 tempest-VolumeQuotaClassesTest-1989696018-project-admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:32.357004 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc1900-216f-4cc8-a81f-790632ab813e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:32.357641 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc1900-216f-4cc8-a81f-790632ab813e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:32.357922 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc1900-216f-4cc8-a81f-790632ab813e admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:32.358194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc1900-216f-4cc8-a81f-790632ab813e admin admin] PATH_INFO: `/v3/users/f3df72de1a3040c1a20aa88e0aa18c3b` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:32.359355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc1900-216f-4cc8-a81f-790632ab813e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3df72de1a3040c1a20aa88e0aa18c3b)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:32.361306 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc1900-216f-4cc8-a81f-790632ab813e admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:32.407865 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-80dc1900-216f-4cc8-a81f-790632ab813e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3df72de1a3040c1a20aa88e0aa18c3b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:32.408543 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-80dc1900-216f-4cc8-a81f-790632ab813e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3df72de1a3040c1a20aa88e0aa18c3b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:32.412947 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:32.423280 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-80dc1900-216f-4cc8-a81f-790632ab813e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3df72de1a3040c1a20aa88e0aa18c3b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:32.430315 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-80dc1900-216f-4cc8-a81f-790632ab813e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3df72de1a3040c1a20aa88e0aa18c3b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:32.438070 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-80dc1900-216f-4cc8-a81f-790632ab813e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3df72de1a3040c1a20aa88e0aa18c3b'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:32.444233 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1279/2533] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 01:06:32 2022] DELETE /identity/v3/users/f3df72de1a3040c1a20aa88e0aa18c3b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:32.449003 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cad877-ccf5-41f9-b12b-d9b3b245f2a9 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:32.449518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cad877-ccf5-41f9-b12b-d9b3b245f2a9 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:32.449702 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cad877-ccf5-41f9-b12b-d9b3b245f2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:32.468702 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-23cad877-ccf5-41f9-b12b-d9b3b245f2a9 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:32.469846 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-23cad877-ccf5-41f9-b12b-d9b3b245f2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:32.528393 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1255/2534] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:32.538329 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc1900-216f-4cc8-a81f-790632ab813e admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:32.539767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-80dc1900-216f-4cc8-a81f-790632ab813e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:32.606731 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc1900-216f-4cc8-a81f-790632ab813e admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:32.610901 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c787c5f-de3e-471b-8684-7476be9aeac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:32.611513 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c787c5f-de3e-471b-8684-7476be9aeac3 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:32.611822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c787c5f-de3e-471b-8684-7476be9aeac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:32.612109 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c787c5f-de3e-471b-8684-7476be9aeac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:32.621306 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c787c5f-de3e-471b-8684-7476be9aeac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:32.622723 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c787c5f-de3e-471b-8684-7476be9aeac3 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:32.632747 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1280/2535] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:06:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:06:32.896982 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cad877-ccf5-41f9-b12b-d9b3b245f2a9 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:32.898559 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-23cad877-ccf5-41f9-b12b-d9b3b245f2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:32.912873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cad877-ccf5-41f9-b12b-d9b3b245f2a9 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:32.917473 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89f51c-3b95-4df2-8437-6f3e14e8da19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:32.918103 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89f51c-3b95-4df2-8437-6f3e14e8da19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:32.918380 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89f51c-3b95-4df2-8437-6f3e14e8da19 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:32.918682 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89f51c-3b95-4df2-8437-6f3e14e8da19 admin admin] PATH_INFO: `/v3/projects/cf7545997eb64ce7897101fac680fd7c` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:32.919799 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89f51c-3b95-4df2-8437-6f3e14e8da19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf7545997eb64ce7897101fac680fd7c)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:32.922479 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89f51c-3b95-4df2-8437-6f3e14e8da19 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:32.971360 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-db89f51c-3b95-4df2-8437-6f3e14e8da19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf7545997eb64ce7897101fac680fd7c', 'request_id': 'req-db89f51c-3b95-4df2-8437-6f3e14e8da19'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:32.979904 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-db89f51c-3b95-4df2-8437-6f3e14e8da19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf7545997eb64ce7897101fac680fd7c', 'request_id': 'req-db89f51c-3b95-4df2-8437-6f3e14e8da19'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:32.987632 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-db89f51c-3b95-4df2-8437-6f3e14e8da19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf7545997eb64ce7897101fac680fd7c', 'request_id': 'req-db89f51c-3b95-4df2-8437-6f3e14e8da19'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:32.998112 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-db89f51c-3b95-4df2-8437-6f3e14e8da19 admin admin] The token cache is being invalidate because project cf7545997eb64ce7897101fac680fd7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:06:32.998398 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-db89f51c-3b95-4df2-8437-6f3e14e8da19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:32.999354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-db89f51c-3b95-4df2-8437-6f3e14e8da19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:33.000870 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1256/2536] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 01:06:32 2022] DELETE /identity/v3/projects/cf7545997eb64ce7897101fac680fd7c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:33.004870 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c787c5f-de3e-471b-8684-7476be9aeac3 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:33.014471 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c787c5f-de3e-471b-8684-7476be9aeac3 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:33.071877 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a734b4-0c0d-4d13-b91b-68b90cd202bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:33.072784 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a734b4-0c0d-4d13-b91b-68b90cd202bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:33.073065 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a734b4-0c0d-4d13-b91b-68b90cd202bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:33.073335 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a734b4-0c0d-4d13-b91b-68b90cd202bf admin admin] PATH_INFO: `/v3/users/b3c4334f2cf4469a8cf04b89f8398e54` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:33.074431 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a734b4-0c0d-4d13-b91b-68b90cd202bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3c4334f2cf4469a8cf04b89f8398e54)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:33.076355 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a734b4-0c0d-4d13-b91b-68b90cd202bf admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:33.122733 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-06a734b4-0c0d-4d13-b91b-68b90cd202bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3c4334f2cf4469a8cf04b89f8398e54'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:33.123393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-06a734b4-0c0d-4d13-b91b-68b90cd202bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3c4334f2cf4469a8cf04b89f8398e54'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:33.135509 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-06a734b4-0c0d-4d13-b91b-68b90cd202bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3c4334f2cf4469a8cf04b89f8398e54'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:33.142925 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-06a734b4-0c0d-4d13-b91b-68b90cd202bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3c4334f2cf4469a8cf04b89f8398e54'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:33.144605 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:33.150485 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-06a734b4-0c0d-4d13-b91b-68b90cd202bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3c4334f2cf4469a8cf04b89f8398e54'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:33.158626 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1281/2537] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 01:06:33 2022] DELETE /identity/v3/users/b3c4334f2cf4469a8cf04b89f8398e54 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:33.431158 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89f51c-3b95-4df2-8437-6f3e14e8da19 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:33.432601 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-db89f51c-3b95-4df2-8437-6f3e14e8da19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:33.446179 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89f51c-3b95-4df2-8437-6f3e14e8da19 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:33.467397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:33.506128 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e756786-a7fe-4a7e-998d-bbe3c3cfb241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:33.506704 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e756786-a7fe-4a7e-998d-bbe3c3cfb241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:33.506986 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e756786-a7fe-4a7e-998d-bbe3c3cfb241 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:33.507262 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e756786-a7fe-4a7e-998d-bbe3c3cfb241 admin admin] PATH_INFO: `/v3/projects/ae759d31f45249c09b4a8dd633570cbc` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:33.508398 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e756786-a7fe-4a7e-998d-bbe3c3cfb241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae759d31f45249c09b4a8dd633570cbc)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:33.510301 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e756786-a7fe-4a7e-998d-bbe3c3cfb241 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:33.555841 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-3e756786-a7fe-4a7e-998d-bbe3c3cfb241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae759d31f45249c09b4a8dd633570cbc', 'request_id': 'req-3e756786-a7fe-4a7e-998d-bbe3c3cfb241'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:33.562797 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-3e756786-a7fe-4a7e-998d-bbe3c3cfb241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae759d31f45249c09b4a8dd633570cbc', 'request_id': 'req-3e756786-a7fe-4a7e-998d-bbe3c3cfb241'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:33.568823 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-3e756786-a7fe-4a7e-998d-bbe3c3cfb241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae759d31f45249c09b4a8dd633570cbc', 'request_id': 'req-3e756786-a7fe-4a7e-998d-bbe3c3cfb241'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:33.580061 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-3e756786-a7fe-4a7e-998d-bbe3c3cfb241 admin admin] The token cache is being invalidate because project ae759d31f45249c09b4a8dd633570cbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:06:33.580377 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-3e756786-a7fe-4a7e-998d-bbe3c3cfb241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:33.581005 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-3e756786-a7fe-4a7e-998d-bbe3c3cfb241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:33.582472 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1257/2538] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 01:06:33 2022] DELETE /identity/v3/projects/ae759d31f45249c09b4a8dd633570cbc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:33.587319 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd7089-6628-4a4f-b8f5-99d5f5d686c1 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:33.587612 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd7089-6628-4a4f-b8f5-99d5f5d686c1 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:33.587893 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd7089-6628-4a4f-b8f5-99d5f5d686c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:33.607264 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-cebd7089-6628-4a4f-b8f5-99d5f5d686c1 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:33.608720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-cebd7089-6628-4a4f-b8f5-99d5f5d686c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:33.667203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1282/2539] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:33.674777 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244b73e-a471-499b-81e4-ab57a4280498 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:33.675067 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244b73e-a471-499b-81e4-ab57a4280498 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:33.675365 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244b73e-a471-499b-81e4-ab57a4280498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:33.695495 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-4244b73e-a471-499b-81e4-ab57a4280498 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:33.696778 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4244b73e-a471-499b-81e4-ab57a4280498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:33.705735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1258/2540] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:06:33.709809 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd7089-6628-4a4f-b8f5-99d5f5d686c1 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:33.719556 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd7089-6628-4a4f-b8f5-99d5f5d686c1 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:33.724068 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a57fd-58d8-40d4-85ac-8446ab77bc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:33.724684 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a57fd-58d8-40d4-85ac-8446ab77bc2b admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:33.724959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a57fd-58d8-40d4-85ac-8446ab77bc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:33.725230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a57fd-58d8-40d4-85ac-8446ab77bc2b admin admin] PATH_INFO: `/v3/domains` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:33.725865 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a57fd-58d8-40d4-85ac-8446ab77bc2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:06:33.726165 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a57fd-58d8-40d4-85ac-8446ab77bc2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:33.727743 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a57fd-58d8-40d4-85ac-8446ab77bc2b admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:33.739111 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1283/2541] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 01:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:06:33.743465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244b73e-a471-499b-81e4-ab57a4280498 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:33.753287 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244b73e-a471-499b-81e4-ab57a4280498 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:33.758021 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4b2b3-a171-40bb-8209-5fe12026573d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:33.758557 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4b2b3-a171-40bb-8209-5fe12026573d admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:33.758839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4b2b3-a171-40bb-8209-5fe12026573d admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:33.759115 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4b2b3-a171-40bb-8209-5fe12026573d admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:33.759735 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4b2b3-a171-40bb-8209-5fe12026573d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:33.761770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4b2b3-a171-40bb-8209-5fe12026573d admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:33.790217 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1259/2542] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 01:06:33 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:06:33.794418 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a57fd-58d8-40d4-85ac-8446ab77bc2b admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:33.805019 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a57fd-58d8-40d4-85ac-8446ab77bc2b admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:33.860835 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0a308-b543-4bf4-a743-dad9415aaed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:33.861433 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0a308-b543-4bf4-a743-dad9415aaed6 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:33.861710 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0a308-b543-4bf4-a743-dad9415aaed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:33.861977 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0a308-b543-4bf4-a743-dad9415aaed6 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:33.862601 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0a308-b543-4bf4-a743-dad9415aaed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:33.864546 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0a308-b543-4bf4-a743-dad9415aaed6 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:33.885915 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1284/2543] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 01:06:33 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:06:33.890296 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4b2b3-a171-40bb-8209-5fe12026573d admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:33.900704 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4b2b3-a171-40bb-8209-5fe12026573d admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:33.905523 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a96c1b-fec4-47e8-8cd0-7683a1460ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:33.906168 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a96c1b-fec4-47e8-8cd0-7683a1460ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:33.906449 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a96c1b-fec4-47e8-8cd0-7683a1460ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:33.906748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a96c1b-fec4-47e8-8cd0-7683a1460ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:33.907390 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a96c1b-fec4-47e8-8cd0-7683a1460ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:33.909069 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a96c1b-fec4-47e8-8cd0-7683a1460ce5 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:33.919020 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1260/2544] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:06:33.923950 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0a308-b543-4bf4-a743-dad9415aaed6 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:33.933810 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0a308-b543-4bf4-a743-dad9415aaed6 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:33.939338 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e426a34-6c3c-4af2-8f8a-3a5b3409e2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:33.939943 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e426a34-6c3c-4af2-8f8a-3a5b3409e2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:33.940281 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e426a34-6c3c-4af2-8f8a-3a5b3409e2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:33.940562 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e426a34-6c3c-4af2-8f8a-3a5b3409e2d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:33.941191 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e426a34-6c3c-4af2-8f8a-3a5b3409e2d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:33.942783 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e426a34-6c3c-4af2-8f8a-3a5b3409e2d1 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:33.954056 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1285/2545] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:06:33.958190 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a96c1b-fec4-47e8-8cd0-7683a1460ce5 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:33.968248 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a96c1b-fec4-47e8-8cd0-7683a1460ce5 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:33.972894 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b07927-1e36-4874-90dc-d3ed3a5b4fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:33.973566 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b07927-1e36-4874-90dc-d3ed3a5b4fd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:33.973843 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b07927-1e36-4874-90dc-d3ed3a5b4fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:33.974115 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b07927-1e36-4874-90dc-d3ed3a5b4fd9 admin admin] PATH_INFO: `/v3/projects/0ead943895cb41ae9b1d825bde1640c6/users/d6443431d84b4922a2b0421fd65169a6/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:33.989152 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b07927-1e36-4874-90dc-d3ed3a5b4fd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ead943895cb41ae9b1d825bde1640c6, user_id=d6443431d84b4922a2b0421fd65169a6, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:33.991787 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b07927-1e36-4874-90dc-d3ed3a5b4fd9 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:34.004906 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1261/2546] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:06:33 2022] PUT /identity/v3/projects/0ead943895cb41ae9b1d825bde1640c6/users/d6443431d84b4922a2b0421fd65169a6/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:34.015483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81093a9-2569-4a3e-a30d-4710722a9cfe None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:34.015978 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81093a9-2569-4a3e-a30d-4710722a9cfe None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:34.015978 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81093a9-2569-4a3e-a30d-4710722a9cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:34.044014 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-a81093a9-2569-4a3e-a30d-4710722a9cfe None None] MFA Rules not processed for user `d6443431d84b4922a2b0421fd65169a6`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:34.045236 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a81093a9-2569-4a3e-a30d-4710722a9cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:34.094486 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:34.115707 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1286/2547] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:34.120241 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b07927-1e36-4874-90dc-d3ed3a5b4fd9 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:34.170256 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:34.183569 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b07927-1e36-4874-90dc-d3ed3a5b4fd9 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:34.189263 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30fe57-0309-427d-8e88-fb520bd2bb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:34.189881 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30fe57-0309-427d-8e88-fb520bd2bb34 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:34.190159 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30fe57-0309-427d-8e88-fb520bd2bb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:34.190431 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30fe57-0309-427d-8e88-fb520bd2bb34 admin admin] PATH_INFO: `/v3/projects` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:34.191048 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30fe57-0309-427d-8e88-fb520bd2bb34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:34.192978 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30fe57-0309-427d-8e88-fb520bd2bb34 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:34.220383 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1262/2548] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 01:06:34 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:06:34.224410 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81093a9-2569-4a3e-a30d-4710722a9cfe None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:34.236403 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81093a9-2569-4a3e-a30d-4710722a9cfe None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:34.292990 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8145e3a-52d7-4f84-9555-8086b55abab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:34.293516 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8145e3a-52d7-4f84-9555-8086b55abab4 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:34.293789 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8145e3a-52d7-4f84-9555-8086b55abab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:34.294058 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8145e3a-52d7-4f84-9555-8086b55abab4 admin admin] PATH_INFO: `/v3/users` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:34.294671 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8145e3a-52d7-4f84-9555-8086b55abab4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:34.297683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8145e3a-52d7-4f84-9555-8086b55abab4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:34.321656 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1287/2549] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 01:06:34 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:06:34.323186 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30fe57-0309-427d-8e88-fb520bd2bb34 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:34.333477 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30fe57-0309-427d-8e88-fb520bd2bb34 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:34.338196 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e465c-ad96-469e-afa8-6e238bc477a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:34.338792 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e465c-ad96-469e-afa8-6e238bc477a9 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:34.339079 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e465c-ad96-469e-afa8-6e238bc477a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:34.339355 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e465c-ad96-469e-afa8-6e238bc477a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:34.339992 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e465c-ad96-469e-afa8-6e238bc477a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:34.341686 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e465c-ad96-469e-afa8-6e238bc477a9 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:34.352697 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1263/2550] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:06:34.356828 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8145e3a-52d7-4f84-9555-8086b55abab4 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:34.367054 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8145e3a-52d7-4f84-9555-8086b55abab4 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:34.371562 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ce640-ebb1-41fb-98f0-65276e0ae406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:34.372232 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ce640-ebb1-41fb-98f0-65276e0ae406 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:34.372508 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ce640-ebb1-41fb-98f0-65276e0ae406 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:34.372780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ce640-ebb1-41fb-98f0-65276e0ae406 admin admin] PATH_INFO: `/v3/projects/ca49b487a241435f8cbeff65baba3940/users/1fa16d8b74f9429cac497e183f127fa5/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:34.387258 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ce640-ebb1-41fb-98f0-65276e0ae406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca49b487a241435f8cbeff65baba3940, user_id=1fa16d8b74f9429cac497e183f127fa5, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:34.390167 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ce640-ebb1-41fb-98f0-65276e0ae406 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:34.402501 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1288/2551] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:06:34 2022] PUT /identity/v3/projects/ca49b487a241435f8cbeff65baba3940/users/1fa16d8b74f9429cac497e183f127fa5/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:34.406931 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e465c-ad96-469e-afa8-6e238bc477a9 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:34.416881 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e465c-ad96-469e-afa8-6e238bc477a9 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:34.473727 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de27c7-2a39-4056-9821-175ceb933358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:34.474403 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de27c7-2a39-4056-9821-175ceb933358 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:34.474598 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de27c7-2a39-4056-9821-175ceb933358 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:34.474873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de27c7-2a39-4056-9821-175ceb933358 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:34.475503 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de27c7-2a39-4056-9821-175ceb933358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:34.477149 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de27c7-2a39-4056-9821-175ceb933358 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:34.487559 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1264/2552] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:06:34.491675 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ce640-ebb1-41fb-98f0-65276e0ae406 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:34.501946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ce640-ebb1-41fb-98f0-65276e0ae406 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:34.506752 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93520f9-f823-418c-96ea-607badb713e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:34.507390 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93520f9-f823-418c-96ea-607badb713e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:34.507667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93520f9-f823-418c-96ea-607badb713e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:34.507936 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93520f9-f823-418c-96ea-607badb713e4 admin admin] PATH_INFO: `/v3/projects/ca49b487a241435f8cbeff65baba3940/users/1fa16d8b74f9429cac497e183f127fa5/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:34.509842 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93520f9-f823-418c-96ea-607badb713e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca49b487a241435f8cbeff65baba3940, user_id=1fa16d8b74f9429cac497e183f127fa5, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:34.512479 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93520f9-f823-418c-96ea-607badb713e4 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:34.522795 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1289/2553] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:06:34 2022] PUT /identity/v3/projects/ca49b487a241435f8cbeff65baba3940/users/1fa16d8b74f9429cac497e183f127fa5/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:34.534583 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43300db-5f6b-43f8-b1d5-e72685347334 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:34.534874 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43300db-5f6b-43f8-b1d5-e72685347334 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:34.535151 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43300db-5f6b-43f8-b1d5-e72685347334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:34.565608 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-b43300db-5f6b-43f8-b1d5-e72685347334 None None] MFA Rules not processed for user `1fa16d8b74f9429cac497e183f127fa5`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:34.566747 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b43300db-5f6b-43f8-b1d5-e72685347334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:34.633793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1265/2554] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:34 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:34.645421 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93520f9-f823-418c-96ea-607badb713e4 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:34.647123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e93520f9-f823-418c-96ea-607badb713e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:34.705036 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93520f9-f823-418c-96ea-607badb713e4 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:34.708675 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e3476d-1414-47ae-b806-e5a259a636b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:34.709273 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3476d-1414-47ae-b806-e5a259a636b5 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:34.709550 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3476d-1414-47ae-b806-e5a259a636b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:34.709822 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3476d-1414-47ae-b806-e5a259a636b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:34.719452 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e3476d-1414-47ae-b806-e5a259a636b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:34.720894 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e3476d-1414-47ae-b806-e5a259a636b5 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:34.730195 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1290/2555] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:06:34 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:06:35.033324 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43300db-5f6b-43f8-b1d5-e72685347334 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:35.084382 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:35.102415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43300db-5f6b-43f8-b1d5-e72685347334 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:35.107033 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439df95-102c-4192-8d30-ea97f44ec3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:35.107666 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439df95-102c-4192-8d30-ea97f44ec3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:35.107946 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439df95-102c-4192-8d30-ea97f44ec3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:35.108320 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439df95-102c-4192-8d30-ea97f44ec3e8 admin admin] PATH_INFO: `/v3/users/d6443431d84b4922a2b0421fd65169a6` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:35.110551 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439df95-102c-4192-8d30-ea97f44ec3e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6443431d84b4922a2b0421fd65169a6)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:35.112994 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439df95-102c-4192-8d30-ea97f44ec3e8 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:35.164863 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e439df95-102c-4192-8d30-ea97f44ec3e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6443431d84b4922a2b0421fd65169a6'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:35.165516 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e439df95-102c-4192-8d30-ea97f44ec3e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6443431d84b4922a2b0421fd65169a6'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:35.181037 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e439df95-102c-4192-8d30-ea97f44ec3e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6443431d84b4922a2b0421fd65169a6'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:35.189118 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e439df95-102c-4192-8d30-ea97f44ec3e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6443431d84b4922a2b0421fd65169a6'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:35.189868 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-e439df95-102c-4192-8d30-ea97f44ec3e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6443431d84b4922a2b0421fd65169a6'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:35.204549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1266/2556] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 01:06:35 2022] DELETE /identity/v3/users/d6443431d84b4922a2b0421fd65169a6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:35.209055 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a980ec9-af8a-4ca0-a487-9dfc90ea1d6c None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:35.209565 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a980ec9-af8a-4ca0-a487-9dfc90ea1d6c None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:35.209682 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a980ec9-af8a-4ca0-a487-9dfc90ea1d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:35.228873 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-3a980ec9-af8a-4ca0-a487-9dfc90ea1d6c None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:35.230092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-3a980ec9-af8a-4ca0-a487-9dfc90ea1d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:35.271445 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:35.294284 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1291/2557] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:06:35.334276 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439df95-102c-4192-8d30-ea97f44ec3e8 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:35.335677 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-e439df95-102c-4192-8d30-ea97f44ec3e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:35.396798 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439df95-102c-4192-8d30-ea97f44ec3e8 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:35.400895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f098e2b-bfa0-44c5-bec7-bf50569aa2ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:35.401241 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f098e2b-bfa0-44c5-bec7-bf50569aa2ee service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:35.401523 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f098e2b-bfa0-44c5-bec7-bf50569aa2ee service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:35.401799 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f098e2b-bfa0-44c5-bec7-bf50569aa2ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:35.411619 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f098e2b-bfa0-44c5-bec7-bf50569aa2ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:35.413087 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f098e2b-bfa0-44c5-bec7-bf50569aa2ee service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:35.425389 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1267/2558] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:06:36.113507 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a980ec9-af8a-4ca0-a487-9dfc90ea1d6c None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:36.114994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-3a980ec9-af8a-4ca0-a487-9dfc90ea1d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:36.127593 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a980ec9-af8a-4ca0-a487-9dfc90ea1d6c None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:36.132178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158c3ca-30d1-4a00-8a57-4f50bb3965de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:36.132872 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158c3ca-30d1-4a00-8a57-4f50bb3965de admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:36.133156 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158c3ca-30d1-4a00-8a57-4f50bb3965de admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:36.133432 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158c3ca-30d1-4a00-8a57-4f50bb3965de admin admin] PATH_INFO: `/v3/projects/0ead943895cb41ae9b1d825bde1640c6` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:36.134563 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158c3ca-30d1-4a00-8a57-4f50bb3965de admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ead943895cb41ae9b1d825bde1640c6)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:36.136515 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158c3ca-30d1-4a00-8a57-4f50bb3965de admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:36.183788 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a158c3ca-30d1-4a00-8a57-4f50bb3965de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ead943895cb41ae9b1d825bde1640c6', 'request_id': 'req-a158c3ca-30d1-4a00-8a57-4f50bb3965de'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:36.191878 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a158c3ca-30d1-4a00-8a57-4f50bb3965de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ead943895cb41ae9b1d825bde1640c6', 'request_id': 'req-a158c3ca-30d1-4a00-8a57-4f50bb3965de'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:36.197903 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a158c3ca-30d1-4a00-8a57-4f50bb3965de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ead943895cb41ae9b1d825bde1640c6', 'request_id': 'req-a158c3ca-30d1-4a00-8a57-4f50bb3965de'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:36.209676 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a158c3ca-30d1-4a00-8a57-4f50bb3965de admin admin] The token cache is being invalidate because project 0ead943895cb41ae9b1d825bde1640c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:06:36.210292 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a158c3ca-30d1-4a00-8a57-4f50bb3965de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:36.210586 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-a158c3ca-30d1-4a00-8a57-4f50bb3965de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:36.212052 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1292/2559] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 01:06:36 2022] DELETE /identity/v3/projects/0ead943895cb41ae9b1d825bde1640c6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:36.216098 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f098e2b-bfa0-44c5-bec7-bf50569aa2ee service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:36.225672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f098e2b-bfa0-44c5-bec7-bf50569aa2ee service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:36.285067 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce5a1f-8643-4e52-ad6b-a731cba7c1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:36.285883 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce5a1f-8643-4e52-ad6b-a731cba7c1c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:36.286164 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce5a1f-8643-4e52-ad6b-a731cba7c1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:36.286436 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce5a1f-8643-4e52-ad6b-a731cba7c1c7 admin admin] PATH_INFO: `/v3/users/1fa16d8b74f9429cac497e183f127fa5` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:36.287553 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce5a1f-8643-4e52-ad6b-a731cba7c1c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fa16d8b74f9429cac497e183f127fa5)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:36.289572 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce5a1f-8643-4e52-ad6b-a731cba7c1c7 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:36.300494 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:36.335810 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-d8ce5a1f-8643-4e52-ad6b-a731cba7c1c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fa16d8b74f9429cac497e183f127fa5'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:36.336640 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-d8ce5a1f-8643-4e52-ad6b-a731cba7c1c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fa16d8b74f9429cac497e183f127fa5'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:36.348973 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-d8ce5a1f-8643-4e52-ad6b-a731cba7c1c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fa16d8b74f9429cac497e183f127fa5'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:36.355677 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-d8ce5a1f-8643-4e52-ad6b-a731cba7c1c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fa16d8b74f9429cac497e183f127fa5'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:36.356349 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-d8ce5a1f-8643-4e52-ad6b-a731cba7c1c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fa16d8b74f9429cac497e183f127fa5'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:36.368526 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1268/2560] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 01:06:36 2022] DELETE /identity/v3/users/1fa16d8b74f9429cac497e183f127fa5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:36.646498 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158c3ca-30d1-4a00-8a57-4f50bb3965de admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:36.648042 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-a158c3ca-30d1-4a00-8a57-4f50bb3965de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:36.660966 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158c3ca-30d1-4a00-8a57-4f50bb3965de admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:36.718340 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc53bd-ad0e-4a20-bf31-baab1c8f8755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:36.719030 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc53bd-ad0e-4a20-bf31-baab1c8f8755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:36.719348 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc53bd-ad0e-4a20-bf31-baab1c8f8755 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:36.719663 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc53bd-ad0e-4a20-bf31-baab1c8f8755 admin admin] PATH_INFO: `/v3/projects/ca49b487a241435f8cbeff65baba3940` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:36.721009 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc53bd-ad0e-4a20-bf31-baab1c8f8755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca49b487a241435f8cbeff65baba3940)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:36.723017 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc53bd-ad0e-4a20-bf31-baab1c8f8755 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:36.769714 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7cfc53bd-ad0e-4a20-bf31-baab1c8f8755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca49b487a241435f8cbeff65baba3940', 'request_id': 'req-7cfc53bd-ad0e-4a20-bf31-baab1c8f8755'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:36.777442 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7cfc53bd-ad0e-4a20-bf31-baab1c8f8755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca49b487a241435f8cbeff65baba3940', 'request_id': 'req-7cfc53bd-ad0e-4a20-bf31-baab1c8f8755'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:36.783208 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7cfc53bd-ad0e-4a20-bf31-baab1c8f8755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca49b487a241435f8cbeff65baba3940', 'request_id': 'req-7cfc53bd-ad0e-4a20-bf31-baab1c8f8755'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:36.785062 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:36.796097 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7cfc53bd-ad0e-4a20-bf31-baab1c8f8755 admin admin] The token cache is being invalidate because project ca49b487a241435f8cbeff65baba3940 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:06:36.797080 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7cfc53bd-ad0e-4a20-bf31-baab1c8f8755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:36.797705 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-7cfc53bd-ad0e-4a20-bf31-baab1c8f8755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:36.799175 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1293/2561] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 01:06:36 2022] DELETE /identity/v3/projects/ca49b487a241435f8cbeff65baba3940 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:36.804319 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aea39e-922c-45aa-acf0-b2858d9ea979 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:36.804610 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aea39e-922c-45aa-acf0-b2858d9ea979 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:36.804914 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aea39e-922c-45aa-acf0-b2858d9ea979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:36.825424 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-49aea39e-922c-45aa-acf0-b2858d9ea979 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:36.826615 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-49aea39e-922c-45aa-acf0-b2858d9ea979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:36.886687 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1269/2562] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:36.894563 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86665fac-04e2-44de-b9d3-605ccc41ca41 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:36.894849 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86665fac-04e2-44de-b9d3-605ccc41ca41 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:36.895125 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86665fac-04e2-44de-b9d3-605ccc41ca41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:36.915884 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-86665fac-04e2-44de-b9d3-605ccc41ca41 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:36.917107 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-86665fac-04e2-44de-b9d3-605ccc41ca41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:36.925649 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1294/2563] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:36.929944 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aea39e-922c-45aa-acf0-b2858d9ea979 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:36.940294 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aea39e-922c-45aa-acf0-b2858d9ea979 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:36.945071 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeebfadb-c72f-480a-b87b-e0b36d9f36ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:36.945667 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeebfadb-c72f-480a-b87b-e0b36d9f36ac admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:36.945947 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeebfadb-c72f-480a-b87b-e0b36d9f36ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:36.946220 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeebfadb-c72f-480a-b87b-e0b36d9f36ac admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:36.947120 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeebfadb-c72f-480a-b87b-e0b36d9f36ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:06:36.947425 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeebfadb-c72f-480a-b87b-e0b36d9f36ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:36.949054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeebfadb-c72f-480a-b87b-e0b36d9f36ac admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:36.960775 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1270/2564] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 01:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:06:36.965061 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86665fac-04e2-44de-b9d3-605ccc41ca41 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:36.974484 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86665fac-04e2-44de-b9d3-605ccc41ca41 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:36.979096 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5369aef-eabf-4238-8335-61727e21f495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:36.979667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5369aef-eabf-4238-8335-61727e21f495 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:36.979944 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5369aef-eabf-4238-8335-61727e21f495 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:36.980252 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5369aef-eabf-4238-8335-61727e21f495 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:36.980885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5369aef-eabf-4238-8335-61727e21f495 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:36.982777 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5369aef-eabf-4238-8335-61727e21f495 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:37.011386 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1295/2565] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 01:06:36 2022] POST /identity/v3/projects => generated 409 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:06:37.015980 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeebfadb-c72f-480a-b87b-e0b36d9f36ac admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:37.025900 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeebfadb-c72f-480a-b87b-e0b36d9f36ac admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:37.083407 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198ba59-8b47-40d1-8029-bd526318149f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:37.084016 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198ba59-8b47-40d1-8029-bd526318149f admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:37.084328 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198ba59-8b47-40d1-8029-bd526318149f admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:37.084604 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198ba59-8b47-40d1-8029-bd526318149f admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:37.085227 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198ba59-8b47-40d1-8029-bd526318149f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:37.087142 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198ba59-8b47-40d1-8029-bd526318149f admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:37.108514 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1271/2566] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 01:06:37 2022] POST /identity/v3/users => generated 305 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:06:37.112869 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5369aef-eabf-4238-8335-61727e21f495 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:37.124582 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5369aef-eabf-4238-8335-61727e21f495 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:37.129212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f290a23-a63a-4d6e-9075-fd2ce85394bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:37.130053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f290a23-a63a-4d6e-9075-fd2ce85394bb admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:37.130293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f290a23-a63a-4d6e-9075-fd2ce85394bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:37.130438 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f290a23-a63a-4d6e-9075-fd2ce85394bb admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:37.131078 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f290a23-a63a-4d6e-9075-fd2ce85394bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:37.132718 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f290a23-a63a-4d6e-9075-fd2ce85394bb admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:37.143160 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1296/2567] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:06:37.147306 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198ba59-8b47-40d1-8029-bd526318149f admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:37.153590 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:37.157703 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198ba59-8b47-40d1-8029-bd526318149f admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:37.163249 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574fd254-2df8-4a31-bafe-e1f291338354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:37.164136 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fd254-2df8-4a31-bafe-e1f291338354 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:37.164592 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fd254-2df8-4a31-bafe-e1f291338354 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:37.165049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fd254-2df8-4a31-bafe-e1f291338354 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:37.165693 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574fd254-2df8-4a31-bafe-e1f291338354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:37.167294 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574fd254-2df8-4a31-bafe-e1f291338354 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:37.178018 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1272/2568] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:06:37.182182 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f290a23-a63a-4d6e-9075-fd2ce85394bb admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:37.192941 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f290a23-a63a-4d6e-9075-fd2ce85394bb admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:37.197801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32209c23-0b3b-4d09-b057-74d7db5ab022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:37.198442 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32209c23-0b3b-4d09-b057-74d7db5ab022 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:37.198721 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32209c23-0b3b-4d09-b057-74d7db5ab022 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:37.198996 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32209c23-0b3b-4d09-b057-74d7db5ab022 admin admin] PATH_INFO: `/v3/projects/a5c412670c6c4eac9d8589100b74eb64/users/0b8642a3eb85452a8215234ead99452e/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:37.214315 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32209c23-0b3b-4d09-b057-74d7db5ab022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5c412670c6c4eac9d8589100b74eb64, user_id=0b8642a3eb85452a8215234ead99452e, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:37.216986 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32209c23-0b3b-4d09-b057-74d7db5ab022 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:37.230020 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1297/2569] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:06:37 2022] PUT /identity/v3/projects/a5c412670c6c4eac9d8589100b74eb64/users/0b8642a3eb85452a8215234ead99452e/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:37.241049 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c6c05-03ca-48a5-9221-895e8f69b009 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:37.241544 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c6c05-03ca-48a5-9221-895e8f69b009 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:37.241659 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c6c05-03ca-48a5-9221-895e8f69b009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:37.269505 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-c21c6c05-03ca-48a5-9221-895e8f69b009 None None] MFA Rules not processed for user `0b8642a3eb85452a8215234ead99452e`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:37.270714 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c21c6c05-03ca-48a5-9221-895e8f69b009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:37.336222 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1273/2570] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:37 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:37.340791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32209c23-0b3b-4d09-b057-74d7db5ab022 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:37.402753 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32209c23-0b3b-4d09-b057-74d7db5ab022 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:37.408286 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a488f9bc-6c6e-4195-a8da-e46dbf039680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:37.408885 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488f9bc-6c6e-4195-a8da-e46dbf039680 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:37.409161 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488f9bc-6c6e-4195-a8da-e46dbf039680 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:37.409431 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488f9bc-6c6e-4195-a8da-e46dbf039680 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:37.410063 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a488f9bc-6c6e-4195-a8da-e46dbf039680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:37.411939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a488f9bc-6c6e-4195-a8da-e46dbf039680 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:37.438849 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1298/2571] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 01:06:37 2022] POST /identity/v3/projects => generated 407 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:06:37.443107 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c6c05-03ca-48a5-9221-895e8f69b009 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:37.454105 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c6c05-03ca-48a5-9221-895e8f69b009 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:37.509318 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909b81d-1a55-4df8-91e6-aabe38499cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:37.509917 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909b81d-1a55-4df8-91e6-aabe38499cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:37.510197 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909b81d-1a55-4df8-91e6-aabe38499cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:37.510466 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909b81d-1a55-4df8-91e6-aabe38499cc2 admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:37.511095 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909b81d-1a55-4df8-91e6-aabe38499cc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:37.513065 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8909b81d-1a55-4df8-91e6-aabe38499cc2 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:37.534325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1274/2572] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 01:06:37 2022] POST /identity/v3/users => generated 310 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:06:37.538427 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a488f9bc-6c6e-4195-a8da-e46dbf039680 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:37.550349 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a488f9bc-6c6e-4195-a8da-e46dbf039680 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:37.554936 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff87af-216e-409b-9f8b-f959bb3a31d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:37.555530 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff87af-216e-409b-9f8b-f959bb3a31d3 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:37.555805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff87af-216e-409b-9f8b-f959bb3a31d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:37.556074 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff87af-216e-409b-9f8b-f959bb3a31d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:37.556792 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff87af-216e-409b-9f8b-f959bb3a31d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:37.558374 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff87af-216e-409b-9f8b-f959bb3a31d3 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:37.568950 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1299/2573] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:06:37.573531 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909b81d-1a55-4df8-91e6-aabe38499cc2 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:37.584625 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909b81d-1a55-4df8-91e6-aabe38499cc2 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:37.589378 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4998a2-544c-4ac7-86b1-7225280603f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:37.590035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4998a2-544c-4ac7-86b1-7225280603f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:37.590312 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4998a2-544c-4ac7-86b1-7225280603f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:37.590586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4998a2-544c-4ac7-86b1-7225280603f4 admin admin] PATH_INFO: `/v3/projects/474481120d264b83baa6187f2535348f/users/85cd93434ae24ff0a5a91996092e419a/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:37.613039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4998a2-544c-4ac7-86b1-7225280603f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=474481120d264b83baa6187f2535348f, user_id=85cd93434ae24ff0a5a91996092e419a, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:37.615848 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4998a2-544c-4ac7-86b1-7225280603f4 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:37.630445 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1275/2574] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:06:37 2022] PUT /identity/v3/projects/474481120d264b83baa6187f2535348f/users/85cd93434ae24ff0a5a91996092e419a/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:37.634625 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff87af-216e-409b-9f8b-f959bb3a31d3 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:37.645994 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff87af-216e-409b-9f8b-f959bb3a31d3 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:37.703566 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63daef80-2486-49d6-ad35-04d4b8495e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:37.704124 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63daef80-2486-49d6-ad35-04d4b8495e53 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:37.704394 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63daef80-2486-49d6-ad35-04d4b8495e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:37.704667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63daef80-2486-49d6-ad35-04d4b8495e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:37.705298 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63daef80-2486-49d6-ad35-04d4b8495e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:37.706913 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63daef80-2486-49d6-ad35-04d4b8495e53 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:37.716790 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1300/2575] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:06:37.721780 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4998a2-544c-4ac7-86b1-7225280603f4 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:37.731997 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4998a2-544c-4ac7-86b1-7225280603f4 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:37.736646 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602306d9-7889-44d2-bd26-ee81b6b3796a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:37.737286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602306d9-7889-44d2-bd26-ee81b6b3796a admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:37.737566 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602306d9-7889-44d2-bd26-ee81b6b3796a admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:37.737841 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602306d9-7889-44d2-bd26-ee81b6b3796a admin admin] PATH_INFO: `/v3/projects/474481120d264b83baa6187f2535348f/users/85cd93434ae24ff0a5a91996092e419a/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:37.739751 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602306d9-7889-44d2-bd26-ee81b6b3796a admin admin] RBAC: Authorizing `identity:create_grant(project_id=474481120d264b83baa6187f2535348f, user_id=85cd93434ae24ff0a5a91996092e419a, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:37.742452 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602306d9-7889-44d2-bd26-ee81b6b3796a admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:37.752514 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1276/2576] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:06:37 2022] PUT /identity/v3/projects/474481120d264b83baa6187f2535348f/users/85cd93434ae24ff0a5a91996092e419a/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:37.763683 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222cfbec-11a8-4092-913e-fbab15512ff6 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:37.763964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222cfbec-11a8-4092-913e-fbab15512ff6 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:37.764912 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222cfbec-11a8-4092-913e-fbab15512ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:37.772256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:37.795063 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-222cfbec-11a8-4092-913e-fbab15512ff6 None None] MFA Rules not processed for user `85cd93434ae24ff0a5a91996092e419a`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:37.796521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-222cfbec-11a8-4092-913e-fbab15512ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:37.864493 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1301/2577] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:37 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:37.875630 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602306d9-7889-44d2-bd26-ee81b6b3796a admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:37.877428 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-602306d9-7889-44d2-bd26-ee81b6b3796a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:37.919292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:37.938564 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602306d9-7889-44d2-bd26-ee81b6b3796a admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:37.942183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef758f-5382-4e82-a6b3-88e29ee68aa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:37.942801 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef758f-5382-4e82-a6b3-88e29ee68aa2 service cinder] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:37.943058 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef758f-5382-4e82-a6b3-88e29ee68aa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:37.943327 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef758f-5382-4e82-a6b3-88e29ee68aa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:37.953097 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef758f-5382-4e82-a6b3-88e29ee68aa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:37.954745 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef758f-5382-4e82-a6b3-88e29ee68aa2 service cinder] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:37.963970 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1277/2578] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:06:37 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:06:38.116945 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222cfbec-11a8-4092-913e-fbab15512ff6 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:38.127496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222cfbec-11a8-4092-913e-fbab15512ff6 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:38.131852 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd746ddd-fc63-4d61-a10c-3ee56ec2cfe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:38.132560 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd746ddd-fc63-4d61-a10c-3ee56ec2cfe3 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:38.132959 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd746ddd-fc63-4d61-a10c-3ee56ec2cfe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:38.133243 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd746ddd-fc63-4d61-a10c-3ee56ec2cfe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:38.191551 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd746ddd-fc63-4d61-a10c-3ee56ec2cfe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:38.193093 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd746ddd-fc63-4d61-a10c-3ee56ec2cfe3 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:38.202906 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1302/2579] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:06:38 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:06:46.561439 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef758f-5382-4e82-a6b3-88e29ee68aa2 service cinder] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:46.622720 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef758f-5382-4e82-a6b3-88e29ee68aa2 service cinder] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:46.627322 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3c94b-565b-4e46-8e9a-0cfd2ffe39ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:46.627942 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3c94b-565b-4e46-8e9a-0cfd2ffe39ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:46.628258 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3c94b-565b-4e46-8e9a-0cfd2ffe39ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:46.628535 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3c94b-565b-4e46-8e9a-0cfd2ffe39ab admin admin] PATH_INFO: `/v3/users/0b8642a3eb85452a8215234ead99452e` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:46.629638 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3c94b-565b-4e46-8e9a-0cfd2ffe39ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b8642a3eb85452a8215234ead99452e)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:46.632689 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3c94b-565b-4e46-8e9a-0cfd2ffe39ab admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:46.679416 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2be3c94b-565b-4e46-8e9a-0cfd2ffe39ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b8642a3eb85452a8215234ead99452e'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:46.680068 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2be3c94b-565b-4e46-8e9a-0cfd2ffe39ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b8642a3eb85452a8215234ead99452e'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:46.691672 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2be3c94b-565b-4e46-8e9a-0cfd2ffe39ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b8642a3eb85452a8215234ead99452e'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:46.698650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2be3c94b-565b-4e46-8e9a-0cfd2ffe39ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b8642a3eb85452a8215234ead99452e'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:46.699288 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-2be3c94b-565b-4e46-8e9a-0cfd2ffe39ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b8642a3eb85452a8215234ead99452e'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:46.711973 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1278/2580] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 01:06:46 2022] DELETE /identity/v3/users/0b8642a3eb85452a8215234ead99452e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:46.716651 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d431a35-6663-47f3-9fc1-9bde53a1828e None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:46.717167 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d431a35-6663-47f3-9fc1-9bde53a1828e None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:46.717289 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d431a35-6663-47f3-9fc1-9bde53a1828e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:46.737283 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-5d431a35-6663-47f3-9fc1-9bde53a1828e None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:46.738488 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-5d431a35-6663-47f3-9fc1-9bde53a1828e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:46.796212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1303/2581] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:46.806669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3c94b-565b-4e46-8e9a-0cfd2ffe39ab admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:46.808069 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-2be3c94b-565b-4e46-8e9a-0cfd2ffe39ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:46.872255 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:46.888946 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3c94b-565b-4e46-8e9a-0cfd2ffe39ab admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:46.893424 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f2d69-3d72-4f39-af12-02d2ecc893f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:46.893987 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f2d69-3d72-4f39-af12-02d2ecc893f2 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:46.894264 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f2d69-3d72-4f39-af12-02d2ecc893f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:46.894533 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f2d69-3d72-4f39-af12-02d2ecc893f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:46.903655 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f2d69-3d72-4f39-af12-02d2ecc893f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:46.905063 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f2d69-3d72-4f39-af12-02d2ecc893f2 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:46.914924 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1279/2582] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:06:47.180563 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d431a35-6663-47f3-9fc1-9bde53a1828e None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:47.182010 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-5d431a35-6663-47f3-9fc1-9bde53a1828e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:47.194279 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d431a35-6663-47f3-9fc1-9bde53a1828e None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:47.199143 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a60b19-f2e8-41bf-b10d-21064de671ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:47.199783 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a60b19-f2e8-41bf-b10d-21064de671ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:47.200066 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a60b19-f2e8-41bf-b10d-21064de671ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:47.200371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a60b19-f2e8-41bf-b10d-21064de671ca admin admin] PATH_INFO: `/v3/projects/a5c412670c6c4eac9d8589100b74eb64` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:47.201496 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a60b19-f2e8-41bf-b10d-21064de671ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5c412670c6c4eac9d8589100b74eb64)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:47.203427 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a60b19-f2e8-41bf-b10d-21064de671ca admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:47.227635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=100 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:47.251462 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-e4a60b19-f2e8-41bf-b10d-21064de671ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5c412670c6c4eac9d8589100b74eb64', 'request_id': 'req-e4a60b19-f2e8-41bf-b10d-21064de671ca'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:47.258429 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-e4a60b19-f2e8-41bf-b10d-21064de671ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5c412670c6c4eac9d8589100b74eb64', 'request_id': 'req-e4a60b19-f2e8-41bf-b10d-21064de671ca'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:47.264106 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-e4a60b19-f2e8-41bf-b10d-21064de671ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5c412670c6c4eac9d8589100b74eb64', 'request_id': 'req-e4a60b19-f2e8-41bf-b10d-21064de671ca'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:47.275716 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-e4a60b19-f2e8-41bf-b10d-21064de671ca admin admin] The token cache is being invalidate because project a5c412670c6c4eac9d8589100b74eb64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:06:47.276002 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-e4a60b19-f2e8-41bf-b10d-21064de671ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:47.276678 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-e4a60b19-f2e8-41bf-b10d-21064de671ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:47.278214 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1304/2583] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 01:06:47 2022] DELETE /identity/v3/projects/a5c412670c6c4eac9d8589100b74eb64 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:47.282341 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f2d69-3d72-4f39-af12-02d2ecc893f2 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:47.291971 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f2d69-3d72-4f39-af12-02d2ecc893f2 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:47.348425 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a41ca0-4f6e-4f82-b626-0d713b3d9e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:47.349053 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a41ca0-4f6e-4f82-b626-0d713b3d9e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:47.349329 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a41ca0-4f6e-4f82-b626-0d713b3d9e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:47.349599 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a41ca0-4f6e-4f82-b626-0d713b3d9e1d admin admin] PATH_INFO: `/v3/users/85cd93434ae24ff0a5a91996092e419a` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:47.350687 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a41ca0-4f6e-4f82-b626-0d713b3d9e1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=85cd93434ae24ff0a5a91996092e419a)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:47.352646 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a41ca0-4f6e-4f82-b626-0d713b3d9e1d admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:47.397777 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-45a41ca0-4f6e-4f82-b626-0d713b3d9e1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85cd93434ae24ff0a5a91996092e419a'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:47.398423 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-45a41ca0-4f6e-4f82-b626-0d713b3d9e1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85cd93434ae24ff0a5a91996092e419a'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:47.410374 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-45a41ca0-4f6e-4f82-b626-0d713b3d9e1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85cd93434ae24ff0a5a91996092e419a'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:47.417080 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-45a41ca0-4f6e-4f82-b626-0d713b3d9e1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85cd93434ae24ff0a5a91996092e419a'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:47.417731 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-45a41ca0-4f6e-4f82-b626-0d713b3d9e1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85cd93434ae24ff0a5a91996092e419a'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:47.430015 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1280/2584] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 01:06:47 2022] DELETE /identity/v3/users/85cd93434ae24ff0a5a91996092e419a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:47.708785 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a60b19-f2e8-41bf-b10d-21064de671ca admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:47.710390 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e4a60b19-f2e8-41bf-b10d-21064de671ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:47.723992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a60b19-f2e8-41bf-b10d-21064de671ca admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:47.783537 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05954e-fc8f-4e5a-9eba-cfb946dd56cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:47.784203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05954e-fc8f-4e5a-9eba-cfb946dd56cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:47.784483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05954e-fc8f-4e5a-9eba-cfb946dd56cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:47.784754 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05954e-fc8f-4e5a-9eba-cfb946dd56cf admin admin] PATH_INFO: `/v3/projects/474481120d264b83baa6187f2535348f` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:47.785860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05954e-fc8f-4e5a-9eba-cfb946dd56cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=474481120d264b83baa6187f2535348f)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:47.788123 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05954e-fc8f-4e5a-9eba-cfb946dd56cf admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:47.835201 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ce05954e-fc8f-4e5a-9eba-cfb946dd56cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '474481120d264b83baa6187f2535348f', 'request_id': 'req-ce05954e-fc8f-4e5a-9eba-cfb946dd56cf'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:47.842416 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ce05954e-fc8f-4e5a-9eba-cfb946dd56cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '474481120d264b83baa6187f2535348f', 'request_id': 'req-ce05954e-fc8f-4e5a-9eba-cfb946dd56cf'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:47.848411 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ce05954e-fc8f-4e5a-9eba-cfb946dd56cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '474481120d264b83baa6187f2535348f', 'request_id': 'req-ce05954e-fc8f-4e5a-9eba-cfb946dd56cf'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:47.859393 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ce05954e-fc8f-4e5a-9eba-cfb946dd56cf admin admin] The token cache is being invalidate because project 474481120d264b83baa6187f2535348f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:06:47.859679 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ce05954e-fc8f-4e5a-9eba-cfb946dd56cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:47.860966 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-ce05954e-fc8f-4e5a-9eba-cfb946dd56cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:06:47.862450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1305/2585] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 01:06:47 2022] DELETE /identity/v3/projects/474481120d264b83baa6187f2535348f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:47.867397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4320e-cd66-4616-8a4c-c9cd5b24a2bd None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:47.867912 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4320e-cd66-4616-8a4c-c9cd5b24a2bd None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:47.868015 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4320e-cd66-4616-8a4c-c9cd5b24a2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:47.888021 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-c8d4320e-cd66-4616-8a4c-c9cd5b24a2bd None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:47.889251 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-c8d4320e-cd66-4616-8a4c-c9cd5b24a2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:47.941850 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:47.952948 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1281/2586] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:47.959946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e31792-d410-44e2-b648-62b94389e1b7 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:47.960273 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e31792-d410-44e2-b648-62b94389e1b7 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:47.960549 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e31792-d410-44e2-b648-62b94389e1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:47.981235 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-37e31792-d410-44e2-b648-62b94389e1b7 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:47.982429 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-37e31792-d410-44e2-b648-62b94389e1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:47.991201 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1306/2587] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:47.995500 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4320e-cd66-4616-8a4c-c9cd5b24a2bd None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:48.006096 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4320e-cd66-4616-8a4c-c9cd5b24a2bd None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:48.010605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac969b-0421-4657-8158-34f921b79a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:48.011194 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac969b-0421-4657-8158-34f921b79a73 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:48.011472 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac969b-0421-4657-8158-34f921b79a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:48.011740 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac969b-0421-4657-8158-34f921b79a73 admin admin] PATH_INFO: `/v3/domains` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:48.012410 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac969b-0421-4657-8158-34f921b79a73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 30 01:06:48.012709 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac969b-0421-4657-8158-34f921b79a73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:48.014293 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac969b-0421-4657-8158-34f921b79a73 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:48.025697 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1282/2588] 10.222.0.36 () {58 vars in 1153 bytes} [Sun Oct 30 01:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 30 01:06:48.029821 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e31792-d410-44e2-b648-62b94389e1b7 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:48.040199 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e31792-d410-44e2-b648-62b94389e1b7 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:48.044789 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7bdaa-22d5-4001-be7b-2270971a69cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:48.045377 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7bdaa-22d5-4001-be7b-2270971a69cd admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:48.045672 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7bdaa-22d5-4001-be7b-2270971a69cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:48.045938 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7bdaa-22d5-4001-be7b-2270971a69cd admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:48.046579 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7bdaa-22d5-4001-be7b-2270971a69cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:48.049656 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7bdaa-22d5-4001-be7b-2270971a69cd admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:48.076791 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1307/2589] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 01:06:48 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:06:48.081178 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac969b-0421-4657-8158-34f921b79a73 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:48.090933 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac969b-0421-4657-8158-34f921b79a73 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:48.148605 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2092b12-bf0d-428c-becb-aa0b7e8ff35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:48.149238 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2092b12-bf0d-428c-becb-aa0b7e8ff35b admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:48.149518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2092b12-bf0d-428c-becb-aa0b7e8ff35b admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:48.149793 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2092b12-bf0d-428c-becb-aa0b7e8ff35b admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:48.150422 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2092b12-bf0d-428c-becb-aa0b7e8ff35b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:48.152365 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2092b12-bf0d-428c-becb-aa0b7e8ff35b admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:48.174156 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1283/2590] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 01:06:48 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 30 01:06:48.178390 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7bdaa-22d5-4001-be7b-2270971a69cd admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:48.189131 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7bdaa-22d5-4001-be7b-2270971a69cd admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:48.193637 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5b508-e606-42bd-9a6b-b935672b3269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:48.194224 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5b508-e606-42bd-9a6b-b935672b3269 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:48.194502 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5b508-e606-42bd-9a6b-b935672b3269 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:48.194771 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5b508-e606-42bd-9a6b-b935672b3269 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:48.195401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5b508-e606-42bd-9a6b-b935672b3269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:48.197268 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5b508-e606-42bd-9a6b-b935672b3269 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:48.202828 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:48.210113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1308/2591] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:06:48.213250 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2092b12-bf0d-428c-becb-aa0b7e8ff35b admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:48.224626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2092b12-bf0d-428c-becb-aa0b7e8ff35b admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:48.229255 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d211ed-353e-48e3-8020-4d9a6151c3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:48.229849 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d211ed-353e-48e3-8020-4d9a6151c3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:48.230130 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d211ed-353e-48e3-8020-4d9a6151c3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:48.230409 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d211ed-353e-48e3-8020-4d9a6151c3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:48.231039 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d211ed-353e-48e3-8020-4d9a6151c3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:48.232685 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d211ed-353e-48e3-8020-4d9a6151c3f6 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:48.244183 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1284/2592] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:06:48.248256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5b508-e606-42bd-9a6b-b935672b3269 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:48.258314 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5b508-e606-42bd-9a6b-b935672b3269 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:48.263154 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c21863-aa13-450b-9908-19acd7638d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:48.263806 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c21863-aa13-450b-9908-19acd7638d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:48.264113 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c21863-aa13-450b-9908-19acd7638d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:48.264391 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c21863-aa13-450b-9908-19acd7638d82 admin admin] PATH_INFO: `/v3/projects/4012d80a074641cc80074df4f602d9f0/users/578ca2679aa44174b912eb70339373cb/roles/2e0593ed85a14197a0eaf37135a01787` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:48.278896 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c21863-aa13-450b-9908-19acd7638d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4012d80a074641cc80074df4f602d9f0, user_id=578ca2679aa44174b912eb70339373cb, role_id=2e0593ed85a14197a0eaf37135a01787)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:48.282132 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c21863-aa13-450b-9908-19acd7638d82 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:48.295872 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1309/2593] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:06:48 2022] PUT /identity/v3/projects/4012d80a074641cc80074df4f602d9f0/users/578ca2679aa44174b912eb70339373cb/roles/2e0593ed85a14197a0eaf37135a01787 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:48.301764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3766b-5829-4ca4-bdf3-c4447177e5d8 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:48.302087 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3766b-5829-4ca4-bdf3-c4447177e5d8 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:48.302354 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3766b-5829-4ca4-bdf3-c4447177e5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:48.321323 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-55b3766b-5829-4ca4-bdf3-c4447177e5d8 None None] MFA Rules not processed for user `245a44aa1d7f43fcad2e727557036aab`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:48.322517 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-55b3766b-5829-4ca4-bdf3-c4447177e5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:48.380518 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1285/2594] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:48.390141 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c21863-aa13-450b-9908-19acd7638d82 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:48.392414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-e6c21863-aa13-450b-9908-19acd7638d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:48.448630 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c21863-aa13-450b-9908-19acd7638d82 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:48.452268 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c112a04-201e-4d4d-8af0-5efa2ac672ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:48.452872 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c112a04-201e-4d4d-8af0-5efa2ac672ec service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:48.453172 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c112a04-201e-4d4d-8af0-5efa2ac672ec service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:48.453974 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c112a04-201e-4d4d-8af0-5efa2ac672ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:48.462783 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c112a04-201e-4d4d-8af0-5efa2ac672ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:48.464164 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c112a04-201e-4d4d-8af0-5efa2ac672ec service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:48.474230 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1310/2595] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:06:52.486202 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147bc349-c02f-4385-b7fa-d68f1a96660a None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:52.486202 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147bc349-c02f-4385-b7fa-d68f1a96660a None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:52.486897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147bc349-c02f-4385-b7fa-d68f1a96660a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:52.515821 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-147bc349-c02f-4385-b7fa-d68f1a96660a None None] MFA Rules not processed for user `578ca2679aa44174b912eb70339373cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:52.517728 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-147bc349-c02f-4385-b7fa-d68f1a96660a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:52.580101 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:52.590809 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1286/2596] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:52.596407 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c112a04-201e-4d4d-8af0-5efa2ac672ec service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:52.608293 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c112a04-201e-4d4d-8af0-5efa2ac672ec service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:52.613675 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4e73f-584d-4541-8921-d01173fca333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:52.614281 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4e73f-584d-4541-8921-d01173fca333 admin admin] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:52.614558 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4e73f-584d-4541-8921-d01173fca333 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:52.614828 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4e73f-584d-4541-8921-d01173fca333 admin admin] PATH_INFO: `/v3/projects` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:52.615461 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4e73f-584d-4541-8921-d01173fca333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:52.617388 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4e73f-584d-4541-8921-d01173fca333 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:52.645857 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1311/2597] 10.222.0.36 () {60 vars in 1153 bytes} [Sun Oct 30 01:06:52 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:06:52.649931 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147bc349-c02f-4385-b7fa-d68f1a96660a None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:52.661580 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147bc349-c02f-4385-b7fa-d68f1a96660a None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:52.717371 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1e7ec-5e0e-4a7e-b3bf-ffdbadee879e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:52.717897 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1e7ec-5e0e-4a7e-b3bf-ffdbadee879e admin admin] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:52.718178 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1e7ec-5e0e-4a7e-b3bf-ffdbadee879e admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:52.718453 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1e7ec-5e0e-4a7e-b3bf-ffdbadee879e admin admin] PATH_INFO: `/v3/users` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:52.719122 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1e7ec-5e0e-4a7e-b3bf-ffdbadee879e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:52.721731 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1e7ec-5e0e-4a7e-b3bf-ffdbadee879e admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:52.743804 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1287/2598] 10.222.0.36 () {60 vars in 1144 bytes} [Sun Oct 30 01:06:52 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 30 01:06:52.747866 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4e73f-584d-4541-8921-d01173fca333 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:52.758483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4e73f-584d-4541-8921-d01173fca333 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:52.763479 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c094e-aab4-4b69-ba2d-2d74786f2c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:52.764100 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c094e-aab4-4b69-ba2d-2d74786f2c3f admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:52.764409 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c094e-aab4-4b69-ba2d-2d74786f2c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:52.764674 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c094e-aab4-4b69-ba2d-2d74786f2c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:52.765306 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c094e-aab4-4b69-ba2d-2d74786f2c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:52.767086 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c094e-aab4-4b69-ba2d-2d74786f2c3f admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:52.777671 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1312/2599] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 30 01:06:52.781955 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1e7ec-5e0e-4a7e-b3bf-ffdbadee879e admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:52.792529 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1e7ec-5e0e-4a7e-b3bf-ffdbadee879e admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:52.797084 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d9ae6-b367-41e2-b5cd-f1a8d02f247f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:52.797731 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d9ae6-b367-41e2-b5cd-f1a8d02f247f admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:52.798614 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d9ae6-b367-41e2-b5cd-f1a8d02f247f admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:52.798876 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d9ae6-b367-41e2-b5cd-f1a8d02f247f admin admin] PATH_INFO: `/v3/projects/664a78a74336428ab4fc6f88bd04a8f0/users/5368aec12f2b4b12b8c8114c23d21b72/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:52.814493 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d9ae6-b367-41e2-b5cd-f1a8d02f247f admin admin] RBAC: Authorizing `identity:create_grant(project_id=664a78a74336428ab4fc6f88bd04a8f0, user_id=5368aec12f2b4b12b8c8114c23d21b72, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:52.817453 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d9ae6-b367-41e2-b5cd-f1a8d02f247f admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:52.832643 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1288/2600] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:06:52 2022] PUT /identity/v3/projects/664a78a74336428ab4fc6f88bd04a8f0/users/5368aec12f2b4b12b8c8114c23d21b72/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 30 01:06:52.836751 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c094e-aab4-4b69-ba2d-2d74786f2c3f admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:52.847116 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c094e-aab4-4b69-ba2d-2d74786f2c3f admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:52.903978 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bd5d7-75a9-4e07-a8ae-9aa77469b6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:52.904623 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bd5d7-75a9-4e07-a8ae-9aa77469b6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:52.904900 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bd5d7-75a9-4e07-a8ae-9aa77469b6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:52.905170 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bd5d7-75a9-4e07-a8ae-9aa77469b6f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:52.905801 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06bd5d7-75a9-4e07-a8ae-9aa77469b6f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:52.907413 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06bd5d7-75a9-4e07-a8ae-9aa77469b6f3 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:52.917110 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1313/2601] 10.222.0.36 () {58 vars in 1122 bytes} [Sun Oct 30 01:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 30 01:06:52.921389 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d9ae6-b367-41e2-b5cd-f1a8d02f247f admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:52.932860 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d9ae6-b367-41e2-b5cd-f1a8d02f247f admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:52.937462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b75f9d-6ed0-4d72-9cc4-3af2fc0517bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:52.938105 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b75f9d-6ed0-4d72-9cc4-3af2fc0517bb admin admin] REQUEST_METHOD: `PUT` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:52.938385 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b75f9d-6ed0-4d72-9cc4-3af2fc0517bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:52.939328 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b75f9d-6ed0-4d72-9cc4-3af2fc0517bb admin admin] PATH_INFO: `/v3/projects/664a78a74336428ab4fc6f88bd04a8f0/users/5368aec12f2b4b12b8c8114c23d21b72/roles/980241d45fdb435d9f32c434057c71e4` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:52.941260 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b75f9d-6ed0-4d72-9cc4-3af2fc0517bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=664a78a74336428ab4fc6f88bd04a8f0, user_id=5368aec12f2b4b12b8c8114c23d21b72, role_id=980241d45fdb435d9f32c434057c71e4)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:52.943889 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b75f9d-6ed0-4d72-9cc4-3af2fc0517bb admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:52.954310 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1289/2602] 10.222.0.36 () {60 vars in 1483 bytes} [Sun Oct 30 01:06:52 2022] PUT /identity/v3/projects/664a78a74336428ab4fc6f88bd04a8f0/users/5368aec12f2b4b12b8c8114c23d21b72/roles/980241d45fdb435d9f32c434057c71e4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:06:57.150448 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c73590-49d5-473f-893a-bd84b817abf9 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:57.151450 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c73590-49d5-473f-893a-bd84b817abf9 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:57.152133 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c73590-49d5-473f-893a-bd84b817abf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:57.194896 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-09c73590-49d5-473f-893a-bd84b817abf9 None None] MFA Rules not processed for user `5368aec12f2b4b12b8c8114c23d21b72`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:57.197344 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-09c73590-49d5-473f-893a-bd84b817abf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:57.255946 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:57.311481 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1314/2603] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:06:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 30 01:06:57.325465 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b75f9d-6ed0-4d72-9cc4-3af2fc0517bb admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:57.327522 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-95b75f9d-6ed0-4d72-9cc4-3af2fc0517bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:57.407774 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b75f9d-6ed0-4d72-9cc4-3af2fc0517bb admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:57.412880 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d4687-98e5-4983-a0aa-1b34973594b0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cf9451f2ef7478589b4a4278cb03252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:57.414356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d4687-98e5-4983-a0aa-1b34973594b0 service glance] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:57.414356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d4687-98e5-4983-a0aa-1b34973594b0 service glance] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:57.414356 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d4687-98e5-4983-a0aa-1b34973594b0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:57.485882 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d4687-98e5-4983-a0aa-1b34973594b0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:57.487556 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d4687-98e5-4983-a0aa-1b34973594b0 service glance] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:57.497694 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1290/2604] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:06:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:06:57.658955 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34ed09-871d-4883-9fcb-e4a2a7891752 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:57.659264 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34ed09-871d-4883-9fcb-e4a2a7891752 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:57.659543 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34ed09-871d-4883-9fcb-e4a2a7891752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:57.724400 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-7c34ed09-871d-4883-9fcb-e4a2a7891752 None None] MFA Rules not processed for user `8a91d3b9c4834d729211ec367e235889`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:06:57.725920 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-7c34ed09-871d-4883-9fcb-e4a2a7891752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:57.808718 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1315/2605] 10.222.0.36 () {58 vars in 1009 bytes} [Sun Oct 30 01:06:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:06:57.859156 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d4687-98e5-4983-a0aa-1b34973594b0 service glance] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:06:57.861292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-7c9d4687-98e5-4983-a0aa-1b34973594b0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:06:57.865847 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:06:57.962035 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d4687-98e5-4983-a0aa-1b34973594b0 service glance] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:06:57.966986 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc7738-2102-48c8-9d4d-9e8539fa3307 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f4c779b8422432fb06cdf996841b9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:06:57.967587 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc7738-2102-48c8-9d4d-9e8539fa3307 service swift] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:06:57.967865 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc7738-2102-48c8-9d4d-9e8539fa3307 service swift] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:06:57.968170 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc7738-2102-48c8-9d4d-9e8539fa3307 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:06:57.976910 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc7738-2102-48c8-9d4d-9e8539fa3307 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:06:57.978491 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc7738-2102-48c8-9d4d-9e8539fa3307 service swift] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:06:57.987016 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1291/2606] 10.222.0.36 () {60 vars in 1384 bytes} [Sun Oct 30 01:06:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 01:07:06.749098 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17b946-464f-45ff-bfa1-6b1d218a6102 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:06.749635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17b946-464f-45ff-bfa1-6b1d218a6102 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:06.749635 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17b946-464f-45ff-bfa1-6b1d218a6102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:06.769111 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-0a17b946-464f-45ff-bfa1-6b1d218a6102 None None] MFA Rules not processed for user `578ca2679aa44174b912eb70339373cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:07:06.770319 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-0a17b946-464f-45ff-bfa1-6b1d218a6102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:06.777269 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1316/2607] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:07:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:07:07.983208 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=40 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:09.727151 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e708c0-280a-4b70-94d4-cb9354d956f1 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:09.727770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e708c0-280a-4b70-94d4-cb9354d956f1 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:09.727770 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e708c0-280a-4b70-94d4-cb9354d956f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:09.747219 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-d6e708c0-280a-4b70-94d4-cb9354d956f1 None None] MFA Rules not processed for user `8a91d3b9c4834d729211ec367e235889`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:07:09.748461 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d6e708c0-280a-4b70-94d4-cb9354d956f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:09.756569 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1292/2608] 10.222.0.36 () {58 vars in 1009 bytes} [Sun Oct 30 01:07:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:07:09.770158 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17b946-464f-45ff-bfa1-6b1d218a6102 None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:07:09.782212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17b946-464f-45ff-bfa1-6b1d218a6102 None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:07:09.787812 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd4a70-6a75-4ad6-9f93-2b458a05968b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f4c779b8422432fb06cdf996841b9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:07:09.788437 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd4a70-6a75-4ad6-9f93-2b458a05968b service swift] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:09.788717 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd4a70-6a75-4ad6-9f93-2b458a05968b service swift] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:09.788987 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd4a70-6a75-4ad6-9f93-2b458a05968b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:09.798018 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd4a70-6a75-4ad6-9f93-2b458a05968b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:07:09.799594 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd4a70-6a75-4ad6-9f93-2b458a05968b service swift] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:07:09.808470 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1317/2609] 10.222.0.36 () {60 vars in 1384 bytes} [Sun Oct 30 01:07:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 01:07:19.741832 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=3 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:19.804705 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=67 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:25.377577 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e708c0-280a-4b70-94d4-cb9354d956f1 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:07:25.378953 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-d6e708c0-280a-4b70-94d4-cb9354d956f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:25.470512 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e708c0-280a-4b70-94d4-cb9354d956f1 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:07:25.475434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5ed6b-b8b9-4b3b-acc8-a93b147ad4ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:07:25.476032 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5ed6b-b8b9-4b3b-acc8-a93b147ad4ed service placement] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:25.476336 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5ed6b-b8b9-4b3b-acc8-a93b147ad4ed service placement] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:25.476606 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5ed6b-b8b9-4b3b-acc8-a93b147ad4ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:25.478284 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-cdd5ed6b-b8b9-4b3b-acc8-a93b147ad4ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:25.489937 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5ed6b-b8b9-4b3b-acc8-a93b147ad4ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:07:25.491511 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5ed6b-b8b9-4b3b-acc8-a93b147ad4ed service placement] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:07:25.501865 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1293/2610] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:07:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:07:35.497027 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=41 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:42.130575 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd4a70-6a75-4ad6-9f93-2b458a05968b service swift] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:07:42.185834 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd4a70-6a75-4ad6-9f93-2b458a05968b service swift] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:07:42.189328 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f98b75-b476-454c-b545-b352f2702268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:07:42.189921 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f98b75-b476-454c-b545-b352f2702268 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:42.190196 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f98b75-b476-454c-b545-b352f2702268 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:42.190464 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f98b75-b476-454c-b545-b352f2702268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:42.199892 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f98b75-b476-454c-b545-b352f2702268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:07:42.201336 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f98b75-b476-454c-b545-b352f2702268 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:07:42.211550 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1318/2611] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:07:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:07:44.006054 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5ed6b-b8b9-4b3b-acc8-a93b147ad4ed service placement] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:07:44.007586 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-cdd5ed6b-b8b9-4b3b-acc8-a93b147ad4ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:44.127726 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5ed6b-b8b9-4b3b-acc8-a93b147ad4ed service placement] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:07:44.135717 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db1a56-d6d2-4317-b3aa-093a45dac1e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:07:44.136383 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db1a56-d6d2-4317-b3aa-093a45dac1e7 service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:07:44.136987 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db1a56-d6d2-4317-b3aa-093a45dac1e7 service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:07:44.136987 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db1a56-d6d2-4317-b3aa-093a45dac1e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:07:44.139097 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-71db1a56-d6d2-4317-b3aa-093a45dac1e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:07:44.152399 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db1a56-d6d2-4317-b3aa-093a45dac1e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:07:44.153959 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db1a56-d6d2-4317-b3aa-093a45dac1e7 service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:07:44.166192 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1294/2612] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:07:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:07:52.206408 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=26 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:07:54.160427 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=43 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:08:34.654297 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f98b75-b476-454c-b545-b352f2702268 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:08:34.663562 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f98b75-b476-454c-b545-b352f2702268 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:08:34.667194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb8747-19f1-494e-904b-af5effdd55d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:08:34.667779 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb8747-19f1-494e-904b-af5effdd55d3 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:08:34.668053 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb8747-19f1-494e-904b-af5effdd55d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:08:34.668347 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb8747-19f1-494e-904b-af5effdd55d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:08:34.670045 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-f8fb8747-19f1-494e-904b-af5effdd55d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:08:34.683084 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb8747-19f1-494e-904b-af5effdd55d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:08:34.684560 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb8747-19f1-494e-904b-af5effdd55d3 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:08:34.695554 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1319/2613] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:08:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:08:37.068295 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db1a56-d6d2-4317-b3aa-093a45dac1e7 service nova] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:08:37.078562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db1a56-d6d2-4317-b3aa-093a45dac1e7 service nova] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:08:37.082218 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732dec49-7738-4ec4-8ca4-ffcbe6b28322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:08:37.082839 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732dec49-7738-4ec4-8ca4-ffcbe6b28322 service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:08:37.083121 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732dec49-7738-4ec4-8ca4-ffcbe6b28322 service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:08:37.083933 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732dec49-7738-4ec4-8ca4-ffcbe6b28322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:08:37.085749 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-732dec49-7738-4ec4-8ca4-ffcbe6b28322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:08:37.102179 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732dec49-7738-4ec4-8ca4-ffcbe6b28322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:08:37.103648 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732dec49-7738-4ec4-8ca4-ffcbe6b28322 service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:08:37.118855 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1295/2614] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:08:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:08:38.225856 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ec597-96a8-4a81-9ce2-39ab6d207ce5 None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:08:38.226383 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ec597-96a8-4a81-9ce2-39ab6d207ce5 None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:08:38.226383 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ec597-96a8-4a81-9ce2-39ab6d207ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:08:38.245086 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-055ec597-96a8-4a81-9ce2-39ab6d207ce5 None None] MFA Rules not processed for user `d6d3fe4f3fbb4dedb7a0703a2bb6884d`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:08:38.246309 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-055ec597-96a8-4a81-9ce2-39ab6d207ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:08:38.302003 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1320/2615] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:08:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:08:47.112197 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=6 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:08:48.295454 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=31 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:09:39.037611 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732dec49-7738-4ec4-8ca4-ffcbe6b28322 service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:09:39.039036 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-732dec49-7738-4ec4-8ca4-ffcbe6b28322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:39.049828 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732dec49-7738-4ec4-8ca4-ffcbe6b28322 service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:09:39.053462 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fac0c-10e7-4992-a297-f699535c4bab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:09:39.054058 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fac0c-10e7-4992-a297-f699535c4bab service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:09:39.054341 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fac0c-10e7-4992-a297-f699535c4bab service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:09:39.054616 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fac0c-10e7-4992-a297-f699535c4bab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:09:39.056299 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-611fac0c-10e7-4992-a297-f699535c4bab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:09:39.066649 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fac0c-10e7-4992-a297-f699535c4bab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:09:39.068108 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fac0c-10e7-4992-a297-f699535c4bab service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:09:39.076724 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1296/2616] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:09:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:09:49.072425 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=6 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:11.573649 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f95630-b109-4731-a516-eb43df6d0c0d None None] REQUEST_METHOD: `POST` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:11.574154 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f95630-b109-4731-a516-eb43df6d0c0d None None] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:11.574256 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f95630-b109-4731-a516-eb43df6d0c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:11.604550 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.auth.core [None req-03f95630-b109-4731-a516-eb43df6d0c0d None None] MFA Rules not processed for user `8a91d3b9c4834d729211ec367e235889`. Rule list: `[]` (Enabled: `True`). {{(pid=88201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:11.605882 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-03f95630-b109-4731-a516-eb43df6d0c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:11.613366 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1321/2617] 10.222.0.36 () {58 vars in 967 bytes} [Sun Oct 30 01:10:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:11.655712 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fac0c-10e7-4992-a297-f699535c4bab service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:11.668312 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fac0c-10e7-4992-a297-f699535c4bab service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:11.673669 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afb317-c7fe-42f2-9ce2-3e105fc7d2fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f4c779b8422432fb06cdf996841b9b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:11.674290 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afb317-c7fe-42f2-9ce2-3e105fc7d2fd service swift] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:11.674595 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afb317-c7fe-42f2-9ce2-3e105fc7d2fd service swift] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:11.674873 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afb317-c7fe-42f2-9ce2-3e105fc7d2fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:11.682820 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afb317-c7fe-42f2-9ce2-3e105fc7d2fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:11.684512 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afb317-c7fe-42f2-9ce2-3e105fc7d2fd service swift] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:11.693912 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1297/2618] 10.222.0.36 () {60 vars in 1384 bytes} [Sun Oct 30 01:10:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 30 01:10:15.482917 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f95630-b109-4731-a516-eb43df6d0c0d None None] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:15.485166 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-03f95630-b109-4731-a516-eb43df6d0c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:15.503281 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f95630-b109-4731-a516-eb43df6d0c0d None None] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:15.509271 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8598142c-6ec4-481e-9116-49c0bc010a33 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:15.509910 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598142c-6ec4-481e-9116-49c0bc010a33 service placement] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:15.510194 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598142c-6ec4-481e-9116-49c0bc010a33 service placement] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:15.510472 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598142c-6ec4-481e-9116-49c0bc010a33 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:15.512673 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-8598142c-6ec4-481e-9116-49c0bc010a33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:15.534745 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8598142c-6ec4-481e-9116-49c0bc010a33 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:15.538075 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8598142c-6ec4-481e-9116-49c0bc010a33 service placement] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:15.567088 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1322/2619] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 30 01:10:19.165683 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afb317-c7fe-42f2-9ce2-3e105fc7d2fd service swift] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:19.182307 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afb317-c7fe-42f2-9ce2-3e105fc7d2fd service swift] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:19.188478 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56187bc2-6455-41f6-8cb9-6fc9ee573039 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:19.189601 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56187bc2-6455-41f6-8cb9-6fc9ee573039 service placement] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:19.189995 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56187bc2-6455-41f6-8cb9-6fc9ee573039 service placement] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:19.190426 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56187bc2-6455-41f6-8cb9-6fc9ee573039 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:19.192945 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-56187bc2-6455-41f6-8cb9-6fc9ee573039 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:19.208958 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56187bc2-6455-41f6-8cb9-6fc9ee573039 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:19.212634 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56187bc2-6455-41f6-8cb9-6fc9ee573039 service placement] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:19.225682 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1298/2620] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:19.236392 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8598142c-6ec4-481e-9116-49c0bc010a33 service placement] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:19.322057 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8598142c-6ec4-481e-9116-49c0bc010a33 service placement] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:19.326668 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cddf8-0aa7-4af3-bea1-26e7fc48d0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:19.327324 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cddf8-0aa7-4af3-bea1-26e7fc48d0bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:19.327695 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cddf8-0aa7-4af3-bea1-26e7fc48d0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:19.328116 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cddf8-0aa7-4af3-bea1-26e7fc48d0bc admin admin] PATH_INFO: `/v3/users/578ca2679aa44174b912eb70339373cb` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:19.329274 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74cddf8-0aa7-4af3-bea1-26e7fc48d0bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=578ca2679aa44174b912eb70339373cb)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:19.331262 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74cddf8-0aa7-4af3-bea1-26e7fc48d0bc admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:19.379404 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d74cddf8-0aa7-4af3-bea1-26e7fc48d0bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '578ca2679aa44174b912eb70339373cb'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.380073 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d74cddf8-0aa7-4af3-bea1-26e7fc48d0bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '578ca2679aa44174b912eb70339373cb'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.394146 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d74cddf8-0aa7-4af3-bea1-26e7fc48d0bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '578ca2679aa44174b912eb70339373cb'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.401046 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d74cddf8-0aa7-4af3-bea1-26e7fc48d0bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '578ca2679aa44174b912eb70339373cb'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.408797 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d74cddf8-0aa7-4af3-bea1-26e7fc48d0bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '578ca2679aa44174b912eb70339373cb'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.414891 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1323/2621] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:19 2022] DELETE /identity/v3/users/578ca2679aa44174b912eb70339373cb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:19.592711 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56187bc2-6455-41f6-8cb9-6fc9ee573039 service placement] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:19.607447 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56187bc2-6455-41f6-8cb9-6fc9ee573039 service placement] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:19.668623 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489f78e-1b02-4343-8975-ecdd5dc7de33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:19.669562 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489f78e-1b02-4343-8975-ecdd5dc7de33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:19.670045 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489f78e-1b02-4343-8975-ecdd5dc7de33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:19.670397 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489f78e-1b02-4343-8975-ecdd5dc7de33 admin admin] PATH_INFO: `/v3/projects/4012d80a074641cc80074df4f602d9f0` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:19.673964 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489f78e-1b02-4343-8975-ecdd5dc7de33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4012d80a074641cc80074df4f602d9f0)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:19.676733 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489f78e-1b02-4343-8975-ecdd5dc7de33 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:19.727787 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6489f78e-1b02-4343-8975-ecdd5dc7de33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4012d80a074641cc80074df4f602d9f0', 'request_id': 'req-6489f78e-1b02-4343-8975-ecdd5dc7de33'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.735439 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6489f78e-1b02-4343-8975-ecdd5dc7de33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4012d80a074641cc80074df4f602d9f0', 'request_id': 'req-6489f78e-1b02-4343-8975-ecdd5dc7de33'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.741889 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6489f78e-1b02-4343-8975-ecdd5dc7de33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4012d80a074641cc80074df4f602d9f0', 'request_id': 'req-6489f78e-1b02-4343-8975-ecdd5dc7de33'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.752344 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6489f78e-1b02-4343-8975-ecdd5dc7de33 admin admin] The token cache is being invalidate because project 4012d80a074641cc80074df4f602d9f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:19.752637 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6489f78e-1b02-4343-8975-ecdd5dc7de33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.753279 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-6489f78e-1b02-4343-8975-ecdd5dc7de33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.754752 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1299/2622] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:19 2022] DELETE /identity/v3/projects/4012d80a074641cc80074df4f602d9f0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:19.758774 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cddf8-0aa7-4af3-bea1-26e7fc48d0bc admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:19.760280 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d74cddf8-0aa7-4af3-bea1-26e7fc48d0bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:19.773473 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cddf8-0aa7-4af3-bea1-26e7fc48d0bc admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:19.822371 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:19.833825 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566fd65-3570-4aa7-b220-db20b592d8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:19.834481 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566fd65-3570-4aa7-b220-db20b592d8f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:19.834754 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566fd65-3570-4aa7-b220-db20b592d8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:19.835021 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566fd65-3570-4aa7-b220-db20b592d8f0 admin admin] PATH_INFO: `/v3/users/5368aec12f2b4b12b8c8114c23d21b72` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:19.836164 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d566fd65-3570-4aa7-b220-db20b592d8f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5368aec12f2b4b12b8c8114c23d21b72)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:19.838083 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d566fd65-3570-4aa7-b220-db20b592d8f0 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:19.887735 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d566fd65-3570-4aa7-b220-db20b592d8f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5368aec12f2b4b12b8c8114c23d21b72'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.888346 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d566fd65-3570-4aa7-b220-db20b592d8f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5368aec12f2b4b12b8c8114c23d21b72'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.904939 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d566fd65-3570-4aa7-b220-db20b592d8f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5368aec12f2b4b12b8c8114c23d21b72'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.912003 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d566fd65-3570-4aa7-b220-db20b592d8f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5368aec12f2b4b12b8c8114c23d21b72'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.919505 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-d566fd65-3570-4aa7-b220-db20b592d8f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5368aec12f2b4b12b8c8114c23d21b72'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:19.925605 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1324/2623] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 01:10:19 2022] DELETE /identity/v3/users/5368aec12f2b4b12b8c8114c23d21b72 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:20.100764 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489f78e-1b02-4343-8975-ecdd5dc7de33 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:20.102266 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-6489f78e-1b02-4343-8975-ecdd5dc7de33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:20.115832 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489f78e-1b02-4343-8975-ecdd5dc7de33 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:20.175893 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c751499-60b7-4fef-b021-7a48319b9457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:20.176626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c751499-60b7-4fef-b021-7a48319b9457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:20.176893 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c751499-60b7-4fef-b021-7a48319b9457 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:20.177171 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c751499-60b7-4fef-b021-7a48319b9457 admin admin] PATH_INFO: `/v3/projects/664a78a74336428ab4fc6f88bd04a8f0` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:20.178292 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c751499-60b7-4fef-b021-7a48319b9457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=664a78a74336428ab4fc6f88bd04a8f0)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:20.180239 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c751499-60b7-4fef-b021-7a48319b9457 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:20.187302 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:20.228603 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1c751499-60b7-4fef-b021-7a48319b9457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '664a78a74336428ab4fc6f88bd04a8f0', 'request_id': 'req-1c751499-60b7-4fef-b021-7a48319b9457'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:20.236440 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1c751499-60b7-4fef-b021-7a48319b9457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '664a78a74336428ab4fc6f88bd04a8f0', 'request_id': 'req-1c751499-60b7-4fef-b021-7a48319b9457'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:20.242660 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1c751499-60b7-4fef-b021-7a48319b9457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '664a78a74336428ab4fc6f88bd04a8f0', 'request_id': 'req-1c751499-60b7-4fef-b021-7a48319b9457'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:20.253601 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1c751499-60b7-4fef-b021-7a48319b9457 admin admin] The token cache is being invalidate because project 664a78a74336428ab4fc6f88bd04a8f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:10:20.253892 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1c751499-60b7-4fef-b021-7a48319b9457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:20.254543 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-1c751499-60b7-4fef-b021-7a48319b9457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:10:20.256007 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1300/2624] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 01:10:20 2022] DELETE /identity/v3/projects/664a78a74336428ab4fc6f88bd04a8f0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:10:29.922667 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=30,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:30.250831 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:46.651859 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566fd65-3570-4aa7-b220-db20b592d8f0 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:46.653280 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-d566fd65-3570-4aa7-b220-db20b592d8f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:46.732122 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566fd65-3570-4aa7-b220-db20b592d8f0 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:46.783682 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b54a3-9fb7-4fe2-b11c-a017f68b1627 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dbe6b3bdfb3405589b8ce5aebebb93e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:46.784414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b54a3-9fb7-4fe2-b11c-a017f68b1627 service cinder] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:46.784720 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b54a3-9fb7-4fe2-b11c-a017f68b1627 service cinder] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:46.785002 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b54a3-9fb7-4fe2-b11c-a017f68b1627 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:46.786736 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-126b54a3-9fb7-4fe2-b11c-a017f68b1627 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:46.847383 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b54a3-9fb7-4fe2-b11c-a017f68b1627 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:46.848899 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b54a3-9fb7-4fe2-b11c-a017f68b1627 service cinder] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:46.857805 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1325/2625] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:56.853711 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=77 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:10:57.964843 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5a6c6-498c-427c-a6bf-891344cd8607 None None] REQUEST_METHOD: `POST` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:57.965441 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5a6c6-498c-427c-a6bf-891344cd8607 None None] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:57.965441 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5a6c6-498c-427c-a6bf-891344cd8607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:57.987366 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.auth.core [None req-b6a5a6c6-498c-427c-a6bf-891344cd8607 None None] MFA Rules not processed for user `d6d3fe4f3fbb4dedb7a0703a2bb6884d`. Rule list: `[]` (Enabled: `True`). {{(pid=88200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 30 01:10:57.988622 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b6a5a6c6-498c-427c-a6bf-891344cd8607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:58.001286 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1301/2626] 10.222.0.36 () {58 vars in 958 bytes} [Sun Oct 30 01:10:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 30 01:10:58.338440 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b54a3-9fb7-4fe2-b11c-a017f68b1627 service cinder] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:58.339964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-126b54a3-9fb7-4fe2-b11c-a017f68b1627 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:58.400032 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b54a3-9fb7-4fe2-b11c-a017f68b1627 service cinder] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:58.403687 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66204a62-0b99-4133-b314-0d1e66a9d5a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:58.404319 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66204a62-0b99-4133-b314-0d1e66a9d5a1 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:58.404598 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66204a62-0b99-4133-b314-0d1e66a9d5a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:58.404870 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66204a62-0b99-4133-b314-0d1e66a9d5a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:58.406600 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-66204a62-0b99-4133-b314-0d1e66a9d5a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:58.417666 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66204a62-0b99-4133-b314-0d1e66a9d5a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:58.419092 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66204a62-0b99-4133-b314-0d1e66a9d5a1 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:58.427869 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1326/2627] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:10:58.917109 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5a6c6-498c-427c-a6bf-891344cd8607 None None] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:10:58.918585 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b6a5a6c6-498c-427c-a6bf-891344cd8607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:58.996729 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5a6c6-498c-427c-a6bf-891344cd8607 None None] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:10:59.002754 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86bdeb-3364-4955-a3f7-a501f670ab56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:10:59.003368 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86bdeb-3364-4955-a3f7-a501f670ab56 service placement] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:10:59.003648 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86bdeb-3364-4955-a3f7-a501f670ab56 service placement] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:10:59.003921 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86bdeb-3364-4955-a3f7-a501f670ab56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:10:59.005717 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-4e86bdeb-3364-4955-a3f7-a501f670ab56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:10:59.018110 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86bdeb-3364-4955-a3f7-a501f670ab56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:10:59.019721 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86bdeb-3364-4955-a3f7-a501f670ab56 service placement] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:10:59.030644 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1302/2628] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:10:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:01.638860 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66204a62-0b99-4133-b314-0d1e66a9d5a1 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:01.640410 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-66204a62-0b99-4133-b314-0d1e66a9d5a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:01.733522 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66204a62-0b99-4133-b314-0d1e66a9d5a1 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:01.743270 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf4815-65a9-471c-9a69-73adc39b3ad6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:01.744257 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf4815-65a9-471c-9a69-73adc39b3ad6 service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:01.744766 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf4815-65a9-471c-9a69-73adc39b3ad6 service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:01.745212 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf4815-65a9-471c-9a69-73adc39b3ad6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:01.747917 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-1bcf4815-65a9-471c-9a69-73adc39b3ad6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:01.767555 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf4815-65a9-471c-9a69-73adc39b3ad6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:01.769303 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf4815-65a9-471c-9a69-73adc39b3ad6 service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:01.781581 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1327/2629] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:09.025676 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=35 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:11.776062 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=60 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:14.205549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86bdeb-3364-4955-a3f7-a501f670ab56 service placement] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:14.229768 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86bdeb-3364-4955-a3f7-a501f670ab56 service placement] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:14.236876 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbad2f9-3bc7-41ba-83a6-1e64f656cd11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:14.237576 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbad2f9-3bc7-41ba-83a6-1e64f656cd11 service nova] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:14.237865 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbad2f9-3bc7-41ba-83a6-1e64f656cd11 service nova] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:14.238144 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbad2f9-3bc7-41ba-83a6-1e64f656cd11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:14.240802 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-1dbad2f9-3bc7-41ba-83a6-1e64f656cd11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:14.257381 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbad2f9-3bc7-41ba-83a6-1e64f656cd11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:14.259143 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbad2f9-3bc7-41ba-83a6-1e64f656cd11 service nova] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:14.271121 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1303/2630] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:16.969964 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf4815-65a9-471c-9a69-73adc39b3ad6 service nova] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:11:16.980509 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf4815-65a9-471c-9a69-73adc39b3ad6 service nova] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:11:16.983983 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee94793-6faf-49bf-b767-72accd76d9f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:11:16.985178 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee94793-6faf-49bf-b767-72accd76d9f5 service neutron] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:11:16.985608 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee94793-6faf-49bf-b767-72accd76d9f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:11:16.985890 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee94793-6faf-49bf-b767-72accd76d9f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:11:16.987643 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-bee94793-6faf-49bf-b767-72accd76d9f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:11:17.058767 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee94793-6faf-49bf-b767-72accd76d9f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:11:17.060250 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee94793-6faf-49bf-b767-72accd76d9f5 service neutron] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:11:17.070692 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1328/2631] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:11:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:11:24.265803 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=6 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:11:27.065846 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=30 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:12:31.360512 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbad2f9-3bc7-41ba-83a6-1e64f656cd11 service nova] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:12:31.361927 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-1dbad2f9-3bc7-41ba-83a6-1e64f656cd11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:31.375997 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbad2f9-3bc7-41ba-83a6-1e64f656cd11 service nova] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:12:31.381058 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f957b-cabb-4792-a2b2-bc0d1097fb5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:12:31.381655 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f957b-cabb-4792-a2b2-bc0d1097fb5f service placement] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:12:31.381938 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f957b-cabb-4792-a2b2-bc0d1097fb5f service placement] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:12:31.382214 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f957b-cabb-4792-a2b2-bc0d1097fb5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:12:31.383895 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-b12f957b-cabb-4792-a2b2-bc0d1097fb5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:12:31.397773 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f957b-cabb-4792-a2b2-bc0d1097fb5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:12:31.399424 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f957b-cabb-4792-a2b2-bc0d1097fb5f service placement] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:12:31.410748 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1304/2632] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:12:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:12:41.406434 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=6 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:13:22.683780 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee94793-6faf-49bf-b767-72accd76d9f5 service neutron] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:13:22.685488 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-bee94793-6faf-49bf-b767-72accd76d9f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:22.704695 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee94793-6faf-49bf-b767-72accd76d9f5 service neutron] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:13:22.710466 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add734b9-da36-4a11-8416-e0afa7ae7127 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adbba7be39a64230bab160ce77ce7586', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:13:22.711087 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add734b9-da36-4a11-8416-e0afa7ae7127 service nova] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:22.711364 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add734b9-da36-4a11-8416-e0afa7ae7127 service nova] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:22.711653 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add734b9-da36-4a11-8416-e0afa7ae7127 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:22.713414 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-add734b9-da36-4a11-8416-e0afa7ae7127 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:22.725997 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add734b9-da36-4a11-8416-e0afa7ae7127 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:13:22.727589 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add734b9-da36-4a11-8416-e0afa7ae7127 service nova] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:13:22.740401 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1329/2633] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:13:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:13:32.737246 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=6 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:13:34.917653 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f957b-cabb-4792-a2b2-bc0d1097fb5f service placement] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:13:34.927549 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f957b-cabb-4792-a2b2-bc0d1097fb5f service placement] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:13:34.931097 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77736146-b579-4fae-8135-f6b68b34255c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb39b6cc97a1456bb1312f49118c6e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:13:34.931713 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77736146-b579-4fae-8135-f6b68b34255c service neutron] REQUEST_METHOD: `GET` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:34.932010 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77736146-b579-4fae-8135-f6b68b34255c service neutron] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:34.932313 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77736146-b579-4fae-8135-f6b68b34255c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:34.934028 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-77736146-b579-4fae-8135-f6b68b34255c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:34.997650 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77736146-b579-4fae-8135-f6b68b34255c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:13:34.999091 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77736146-b579-4fae-8135-f6b68b34255c service neutron] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:13:35.009642 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1305/2634] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:13:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:13:41.421446 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add734b9-da36-4a11-8416-e0afa7ae7127 service nova] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:13:41.423195 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-add734b9-da36-4a11-8416-e0afa7ae7127 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:41.437633 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add734b9-da36-4a11-8416-e0afa7ae7127 service nova] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:13:41.442624 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38450282-6702-44c1-9eb1-4823912417e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:13:41.443252 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38450282-6702-44c1-9eb1-4823912417e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:41.443782 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38450282-6702-44c1-9eb1-4823912417e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:41.444051 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38450282-6702-44c1-9eb1-4823912417e8 admin admin] PATH_INFO: `/v3/users/d6d3fe4f3fbb4dedb7a0703a2bb6884d` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:41.445191 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38450282-6702-44c1-9eb1-4823912417e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6d3fe4f3fbb4dedb7a0703a2bb6884d)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:13:41.447133 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38450282-6702-44c1-9eb1-4823912417e8 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:13:41.496594 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-38450282-6702-44c1-9eb1-4823912417e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6d3fe4f3fbb4dedb7a0703a2bb6884d'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:41.497251 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-38450282-6702-44c1-9eb1-4823912417e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6d3fe4f3fbb4dedb7a0703a2bb6884d'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:41.509756 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-38450282-6702-44c1-9eb1-4823912417e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6d3fe4f3fbb4dedb7a0703a2bb6884d'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:41.516940 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-38450282-6702-44c1-9eb1-4823912417e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6d3fe4f3fbb4dedb7a0703a2bb6884d'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:41.526991 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-38450282-6702-44c1-9eb1-4823912417e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6d3fe4f3fbb4dedb7a0703a2bb6884d'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:41.535203 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1330/2635] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 01:13:41 2022] DELETE /identity/v3/users/d6d3fe4f3fbb4dedb7a0703a2bb6884d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:13:41.722877 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77736146-b579-4fae-8135-f6b68b34255c service neutron] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:13:41.734018 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77736146-b579-4fae-8135-f6b68b34255c service neutron] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:13:41.795676 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d03ba-15ca-4da0-a3ba-289f9550c074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:13:41.796376 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d03ba-15ca-4da0-a3ba-289f9550c074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:41.796681 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d03ba-15ca-4da0-a3ba-289f9550c074 admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:41.796981 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d03ba-15ca-4da0-a3ba-289f9550c074 admin admin] PATH_INFO: `/v3/projects/a8a24adf04bb4adfbc7347a5e4ec6b52` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:41.798096 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d03ba-15ca-4da0-a3ba-289f9550c074 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8a24adf04bb4adfbc7347a5e4ec6b52)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:13:41.800325 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d03ba-15ca-4da0-a3ba-289f9550c074 admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:13:41.848472 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-244d03ba-15ca-4da0-a3ba-289f9550c074 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8a24adf04bb4adfbc7347a5e4ec6b52', 'request_id': 'req-244d03ba-15ca-4da0-a3ba-289f9550c074'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:41.858683 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-244d03ba-15ca-4da0-a3ba-289f9550c074 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8a24adf04bb4adfbc7347a5e4ec6b52', 'request_id': 'req-244d03ba-15ca-4da0-a3ba-289f9550c074'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:41.865461 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-244d03ba-15ca-4da0-a3ba-289f9550c074 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8a24adf04bb4adfbc7347a5e4ec6b52', 'request_id': 'req-244d03ba-15ca-4da0-a3ba-289f9550c074'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:41.876188 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-244d03ba-15ca-4da0-a3ba-289f9550c074 admin admin] The token cache is being invalidate because project a8a24adf04bb4adfbc7347a5e4ec6b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:13:41.876979 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-244d03ba-15ca-4da0-a3ba-289f9550c074 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:41.877786 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-244d03ba-15ca-4da0-a3ba-289f9550c074 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:41.879584 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1306/2636] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 01:13:41 2022] DELETE /identity/v3/projects/a8a24adf04bb4adfbc7347a5e4ec6b52 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:13:41.883598 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38450282-6702-44c1-9eb1-4823912417e8 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:13:41.885042 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-38450282-6702-44c1-9eb1-4823912417e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:41.899117 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38450282-6702-44c1-9eb1-4823912417e8 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:13:41.956677 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215b60b-a9f4-444c-868b-6572f1ac4116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:13:41.957240 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215b60b-a9f4-444c-868b-6572f1ac4116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:41.957521 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215b60b-a9f4-444c-868b-6572f1ac4116 admin admin] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:41.957798 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215b60b-a9f4-444c-868b-6572f1ac4116 admin admin] PATH_INFO: `/v3/users/9d35d370e5fe4fe9955c348319c24103` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:41.958935 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215b60b-a9f4-444c-868b-6572f1ac4116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d35d370e5fe4fe9955c348319c24103)` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:13:41.960963 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215b60b-a9f4-444c-868b-6572f1ac4116 admin admin] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:13:42.007360 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3215b60b-a9f4-444c-868b-6572f1ac4116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d35d370e5fe4fe9955c348319c24103'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:42.008031 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3215b60b-a9f4-444c-868b-6572f1ac4116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d35d370e5fe4fe9955c348319c24103'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:42.020969 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3215b60b-a9f4-444c-868b-6572f1ac4116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d35d370e5fe4fe9955c348319c24103'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:42.027818 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3215b60b-a9f4-444c-868b-6572f1ac4116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d35d370e5fe4fe9955c348319c24103'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:42.029483 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:13:42.036436 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.notifications [None req-3215b60b-a9f4-444c-868b-6572f1ac4116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d35d370e5fe4fe9955c348319c24103'} {{(pid=88201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:42.043200 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1331/2637] 10.222.0.36 () {58 vars in 1224 bytes} [Sun Oct 30 01:13:41 2022] DELETE /identity/v3/users/9d35d370e5fe4fe9955c348319c24103 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:13:42.216411 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d03ba-15ca-4da0-a3ba-289f9550c074 admin admin] Authenticating user token {{(pid=88200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:13:42.217836 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.fernet_utils [None req-244d03ba-15ca-4da0-a3ba-289f9550c074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:13:42.232105 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d03ba-15ca-4da0-a3ba-289f9550c074 admin admin] Validating token access rules against request {{(pid=88200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:13:42.254713 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:13:42.293132 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44286d55-c924-411a-9247-b66da26372ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '245a44aa1d7f43fcad2e727557036aab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2609b991ac4a86bca190e144b13290', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:13:42.293817 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44286d55-c924-411a-9247-b66da26372ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:13:42.294098 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44286d55-c924-411a-9247-b66da26372ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:13:42.294405 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44286d55-c924-411a-9247-b66da26372ce admin admin] PATH_INFO: `/v3/projects/0c9825d2e86e4f268a95aad241d43c3e` {{(pid=88200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:13:42.295517 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44286d55-c924-411a-9247-b66da26372ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c9825d2e86e4f268a95aad241d43c3e)` {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:13:42.297471 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44286d55-c924-411a-9247-b66da26372ce admin admin] RBAC: Authorization granted {{(pid=88200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:13:42.344626 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-44286d55-c924-411a-9247-b66da26372ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c9825d2e86e4f268a95aad241d43c3e', 'request_id': 'req-44286d55-c924-411a-9247-b66da26372ce'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:42.351983 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-44286d55-c924-411a-9247-b66da26372ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c9825d2e86e4f268a95aad241d43c3e', 'request_id': 'req-44286d55-c924-411a-9247-b66da26372ce'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:42.358747 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-44286d55-c924-411a-9247-b66da26372ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c9825d2e86e4f268a95aad241d43c3e', 'request_id': 'req-44286d55-c924-411a-9247-b66da26372ce'} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:42.372110 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-44286d55-c924-411a-9247-b66da26372ce admin admin] The token cache is being invalidate because project 0c9825d2e86e4f268a95aad241d43c3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 30 01:13:42.372415 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-44286d55-c924-411a-9247-b66da26372ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:42.373064 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG keystone.notifications [None req-44286d55-c924-411a-9247-b66da26372ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 30 01:13:42.374575 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: [pid: 88200|app: 0|req: 1307/2638] 10.222.0.36 () {58 vars in 1233 bytes} [Sun Oct 30 01:13:42 2022] DELETE /identity/v3/projects/0c9825d2e86e4f268a95aad241d43c3e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 30 01:13:52.040043 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:13:52.368945 dv-cinder-861491-6-iscsi devstack@keystone.service[88200]: DEBUG dbcounter [-] [88200] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 30 01:17:35.331042 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215b60b-a9f4-444c-868b-6572f1ac4116 admin admin] Authenticating user token {{(pid=88201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 30 01:17:35.333186 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-3215b60b-a9f4-444c-868b-6572f1ac4116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:17:35.567264 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215b60b-a9f4-444c-868b-6572f1ac4116 admin admin] Validating token access rules against request {{(pid=88201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 30 01:17:35.663196 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71e841-a53d-4226-b89a-b670714623be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ded89d48d6c4a15a5c5829f594f8776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab3a0e5fc5a344739b58ec8313639496', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 30 01:17:35.664253 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71e841-a53d-4226-b89a-b670714623be service placement] REQUEST_METHOD: `GET` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 30 01:17:35.664717 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71e841-a53d-4226-b89a-b670714623be service placement] SCRIPT_NAME: `/identity` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 30 01:17:35.665161 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71e841-a53d-4226-b89a-b670714623be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 30 01:17:35.668560 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.fernet_utils [None req-ea71e841-a53d-4226-b89a-b670714623be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 30 01:17:35.688992 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea71e841-a53d-4226-b89a-b670714623be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 30 01:17:35.691905 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea71e841-a53d-4226-b89a-b670714623be service placement] RBAC: Authorization granted {{(pid=88201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 30 01:17:35.710362 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: [pid: 88201|app: 0|req: 1332/2639] 10.222.0.36 () {60 vars in 1365 bytes} [Sun Oct 30 01:17:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 30 01:17:45.702224 dv-cinder-861491-6-iscsi devstack@keystone.service[88201]: DEBUG dbcounter [-] [88201] Writing DB stats keystone:SELECT=89 {{(pid=88201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}